[RFC] F-Droid source mirror refactoring

[pfalconMember]

What I was trying to suggest was:
…

Sure, that’s what we’re talking about (as sane alternative to do same change in 4+ places). The matter, done that way, it essentially reverts last CiaranG’s commits to his previous implementation, so I’d like to be sure that we considered it from different sides and understand and agree that it’s indeed the right way to do (as part of overall mirroring improvement), and that this thread serves as a good reference to avoid code flapping in the future.

[pfalconMember]

I think the main point is that if you think that we will have to patch the majority (or at least a significant share) of the apps in the repository or not. I hope not.

Then let me tell you something. I wanted to install a map viewer on my Nook Tablet. Obviously, it doesn’t have builtin GPS, but again, I want map viewer, not a navigation. But obviously, any navigation app can serve as map viewer. So, out of dozen navigation apps in F-Droid, none – NONE – were installable on Nook. That’s because their manifest list request for GPS permission. They however forgot to list that GPS is optional. Not patch you say? But then I can’t use the software. Submit upstream? It’s a bit hard to submit to dozen of projects at the same time. Besides, I cannot provide patch for their lack of knowledge of Android.

But if course, I still submit patches, because, well that’s social contract of Open Source. And here’s typical conversation (not GPS, camera this time): “Hi, your app can’t be install on device without camera, here’s the fix” – “Hey, what are you talking about, Android phone cannot be without a camera, god created them such.” Phone? (facepalm) How this is a phone?

tl;dr: State of Android software is very, very poor. Fighting against the tide makes no sense. Massive local patching is the only way to be build kingdom of heaven.

[CiaranGMember]

I agree about the state of a lot of software, but that particular conversation seems like a counter-example to your argument. You pointed out the problem, he now understands, and he says he’s applying your patch in the next release. Sounds like a win to me.

[pfalconMember]

Of course that’s win. Surprise, but there can be lose’s too. And it’s exhausting – to argue to people obvious things: that Android is not just about phones, that it’s their best interest to have the app run on as many devices as possible, and that they should read platform docs. I don’t really have time for that ;-(. And even with all that win, I have no idea when new working version will come out. While I want it working yesterday – not one app, all of them.

[CiaranGMember]

Sure, that’s what we’re talking about (as sane alternative to do same change in 4+ places). The matter, done that way, it essentially reverts last CiaranG’s commits to his previous implementation, so I’d like to be sure that we considered it from different sides and understand and agree that it’s indeed the right way to do (as part of overall mirroring improvement), and that this thread serves as a good reference to avoid code flapping in the future.

I’m still not convinced by this – you’re still talking about it as if we’re trying to model the interface to a vcs, which we’re not. It would be more obvious if the class were not wrongly called ‘vcs’ – but the real function of that class is “get me the source code”. Perhaps the example of applications that do not use a vcs (e.g. tarballs only) would make that clearer?

All I really care about though, is the interface that the other scripts see. As long as they can say “get me the source code” (a.k.a gotorevision() as they now can, I don’t care if it’s trying to model the vcs at a lower level. Previously, multiple scripts (e.g. build, scanner) had to call the right vcs functions in the right order, and conditionally do various things. That was wrong. The right interface is “get me the source code for this revision”, and any other logic is encapsulated in a single place. As long as we don’t take a backwards step there, I’m happy.

The algorithm described in #1432 sounds fine to me. I still have the following reservations:

a) I still maintain that we already have the source mirrored locally, so this basically just doubles the disk space required, for the sake of being able to build when the target repo is offline
b) It doesn’t work for tarballs – I am not too bothered about this really – I tend to think that if a project doesn’t use a version control system, there is something wrong
c) It doesn’t work for svn. Although we can convert 99% of our svn access to git-svn, there are some I think are problematic. Funambol, for example.

I don’t feel strongly enough about any of these things to let them block you from implementing this though.

[pfalconMember]

Perhaps the example of applications that do not use a vcs (e.g. tarballs only) would make that clearer?

Yes, it does. But then we should just have abstract class (well, interface) SourceFetcher with that one method gimme_source(). Tarball fetcher would subclass from this directly. There will be another subclass, VCS, which would implement gimme_source() in terms of abstract vcs methods, to be subclassed and implemented by specific VCS.

As long as they can say “get me the source code” (a.k.a gotorevision() as they now can, I don’t care if it’s trying to model the vcs at a lower level.

Sure, this will hold. (Might need to think what args it gets though – the most generic way is to just pass a build object).

a) I still maintain that we already have the source mirrored locally, so this basically just doubles the disk space required, for the sake of being able to build when the target repo is offline

Well, build dir then can be seen as what it is – scratch area, for example can be removed after a build is done by default (need to add –keep switch to make patching easier).

b) It doesn’t work for tarballs

Well, tarballs would need to be taken into consideration when implementing this, maybe it would be good time to add support for them.

c) It doesn’t work for svn.

That’s true. My original solution to that was allowing to specify repo on a per-build basis. And thinking wide, that’s good thing to have in general. But I wouldn’t tie this with source mirroring, I hope that git-svn indeed should solve majority of practical cases.

I don’t feel strongly enough about any of these things to let them block you from implementing this though.

Thanks. So, I have this in my queue, but probably going to concentrating on adding recipes for near time.

[pfalconMember]


--- a/metadata/com.bwx.bequick.txt
+++ b/metadata/com.bwx.bequick.txt
@@ -17,8 +17,9 @@ Repo:http://quick-settings.googlecode.com/svn/trunk/quick-settings/
Build Version:1.9.8 p2,201012060,184,target=android-8
Build Version:1.9.9.2,201106160,213,target=android-8
Build Version:1.9.9.3,201107260,220,target=android-8
+Build Version:1.9.9.7,201202290,!Source has disappeared


That’s what I mean, why we need to have *proper* mirror. Any upstream may be gone any time. Also, *all* (or large part) of upstreams may be gone anytime. Google just will announce close-down of Google Code tomorrow, like they did with Wave, PowerMeter, and bunch of other services – and oops.

[pfalconMember]

Google just will announce close-down of Google Code tomorrow,

Don’t believe this is possible? Look, it disintegrates in realtime: http://code.google.com/p/support/issues/detail?id=24324 . Just few days ago it was possible in one click to assess how dead a project is, and now – “We are focusing our efforts on features we think will have more impact.” googleplus and other ways to dumb out the population, I guess. Please star it – b$%^%$^ even closed comments there.

[Author]

Posts