Re: [Tinycc-devel] Re: TinyCC CVS, Rob's fork, the future (was: [PATCH 5/5] Fix gv for long longs)

[Rob Landley]

On Sunday 07 September 2008 12:50:58 Daniel Glöckner wrote:
> Today I wrote a shell script that parses the output of "cvs log" and
> generates a diff for every commit. It prepends the log message and a
> list of affected files. There were 47 commits after your automated
> import to Mercurial. If one day you feel like syncing your fork, you
> can use the attached script or just use the tar ball of the diffs at
> http://www.stud.uni-hannover.de/~daniel/tcc/tcccvs-20060220-20080907.tar.bz
>2 (I can only guarantee for this homepage to exist for the next two months.)

The thing is, I'm tired of stopgaps.  I could roll the boulder up the hill one 
more time, but I've synced my fork multiple times, and it just falls out of 
sync again.

I'm tired of maintaining _a_fork_.  I'm tired of my gut response to other 
people's checkins being dread (ala "darn it, more work for me to reverse 
engineer this") rather than happiness that the project has been made better.

And I'm really tired of banging on a project nobody else cares about.  However 
intermittent, this is the mailing list where the users and other developers 
are.  There were once valuable contributors here (you, grischka, David 
Wheeler) doing things I either don't know how to do or will probably never 
get around to doing.  Plus there are still almost twice as many bug reports 
posted here than get emailed to me directly.  (Add in the various distro 
bugzilla variants I've checked for tcc bug reports and this list falls to 
below half of the bug reports I find, although not by much.)

But those other developers have chosen CVS.  And given a choice between 
dealing with CVS and not working on the project at all, it's no 
contest.  "CVS" and "hobby" do not show up in the same sentence for me.

> > It's a question of KILLING OFF THE CVS ARCHIVE, which will never happen.
>
> Well, Savannah does have Subversion, Git, and Mercurial.
> If we bug the right people, the repository might get converted.

I'm all for converting savannah to mercurial.  I happily vote in favor of this 
if anybody's taking votes.  I do not, however, have the authority to do this 
thing, and never have.

Fabrice's original "new maintainer" call was in terms of CVS access:
http://lists.gnu.org/archive/html/tinycc-devel/2006-09/msg00024.html

Almost immediately, he explicitly dismissed a change of source control type as 
being important:
http://lists.gnu.org/archive/html/tinycc-devel/2006-09/msg00058.html

I stopped particularly expecting a move off CVS to ever happen back in 
February 2007, after I cc'd Fabrice on this email:

Rob Landley wrote:
> On Wednesday 21 February 2007 1:56 pm, Geert Janssen wrote:
>> Dear Rob,
>>
>> Are you the tcc maintainer?
> 
> Dunno.  Ask Fabrice.
> 
> I maintain my own mercurial repository, which I converted from the CVS 
> repository back on October 7th (at which point CVS hadn't been touched since 
> February).  I've applied a couple dozen patches to it since then, and have 
at 
> least a dozen more todo items I should make time for.
> 
> However, on October 16th and 18th, Fabrice showed up after a long absence 
and 
> committed some stuff to the old CVS repository (in both cases, variants of 
> patches I think I'd already committed to my repository, the -E thing and 
> stuff from Dave Dodge's patch list), which took most of the wind out of my 
> sails.
> 
> This put my mercurial repository in an awkward position, as a clearly 
> unofficial branch.  If Fabrice showed any desire to take an interest in the 
> project again, I wasn't going to stand in his way, so I stopped actively 
> working on my version at that point.  Went on to focus on other things 
> (toybox and firmware linux, mostly).
> 
> However, it's now been about 4 months since Fabrice's last commit, and he 
> hasn't even tried to catch up with my tree.  I'm still not puting much time 
> into it, and I really haven't properly come up to speed on most of the tcc 
> internals, but I've been bookmarking various tcc patches as I find them and 
> I'm slowly working my way through the backlog.
> 
> So all I can say right now is I maintain _my_ version.  I merge patches and 
> try to respond to bug reports.
> 
> I'm not planning a release (unless somebody really wants one), I have 
several 
> other demands on my time, I really don't understand the internals of the 
> program half as well as I'd like, and I haven't currently got the three 
solid 
> months to dedicate to coming up to speed to my satisfaction.
> 
> I doubt this answers your question.  I think I'm maintaining a fork.
> 
>> If so, I stumbled on some problems too.
> 
> Sure, I'm all ears.
> 
>> tcc had trouble with the struct tags being
>> used at nested scope levels.
> 
> Do you have a code snippet that can reproduce the problem?
> 
>> Also, I have a program that compiles and runs
>> fine on multiple architectures but with tcc
>> is simply hangs.
> 
> Again, got something I can reproduce?  (If I can reproduce it, I'll at least 
> _try_ to fix it, if somebody reminds me often enough. :)
> 
> A couple weeks back somebody sent me a program that segfaults tcc during the 
> compile.  Looking into that is on my short-term todo list (at 
> http://landley.net/notes.html).
> 
>> Geert
> 
> Rob

Which resulted in this exchange:

> Re: [Tinycc-devel] How do I push unsigned with vpushi()?
> From: Fabrice Bellard <address@hidden>
> To: Rob Landley <address@hidden>
> Date: Mon Feb 26 13:05:25 2007
>
> Rob Landley wrote:
> > On Thursday 22 February 2007 4:29 pm, Fabrice Bellard wrote:
> >> Hi Rob,
> >>
> >> I can give you CVS write access to the TinyCC repository on Savannah so
> >> that you can commit the patches you want. Do you have a Savannah account
> >> ?
> >
> > I don't have a savannah account, but I also haven't used CVS in years
> > (except to do read-only anonymous checkouts).  Most of my systems don't
> > even have it installed.
> >
> > I did SVN for busybox, but most of my development these days is done on
> > my laptop, and that's only connected to the net about half the time.
> >  This is why I use distributed source control these days (generally
> > mercurial but I've been meaning to learn git) so I can commit into my
> > laptop repository and rsync it next time I connect.
> >
> > Oh, and my repository already has a number of file renames in it.  SVN
> > could probably handle that, but last I checked CVS still can't.
>
> I prefer to keep the project on Savannah. If it is possible to switch to
> SVN then it is perfectly OK for me. Anyway, most of the TCC code is
> concentrated in less than ten files, so CVS should not be more difficult
> to handle than another tool.
>
> If you don't want to have CVS write access on Savannah, please stop
> complaining about your repository not being in sync :-)
>
> Regards,
>
> Fabrice.

At the time, it was not yet possible to switch Savannah to subversion.  (They 
were talking about it, but it wasn't ready.)  I didn't want CVS write access, 
and had rather a lot of existing work in mercurial, so I promoted my fork to 
a separate project, entirely to avoid having to deal with CVS.

I started my mercurial archive by collecting patches other people had posted 
to the list, which had never been applied to the repository.  Posting my own 
patches to the list for other people to commit obviously wasn't a useful 
option.

My _entire_fork_ has been about avoiding CVS.  Struggling to render other 
developers' work irrelevant is not my idea of fun, but making CVS _GO_AWAY_ 
was, for me, a prerequisite to working on the project.

The CVS archive won.  I quit.

> > Nobody but me seems to be working on an x86-64 target, or even an x86-64
> > _host_.
>
> x86-64 as a target might be problematic because tinycc expects long
> longs to be distributed over two registers.

Nah, "long long" is a red herring.  On x86-64, "long long" can be equal 
to "long".  (Nothing says that sizeof(long long) has to be > 64 bits, it just 
has to  be >= sizeof(long), and == satisfies that constraint.  Note that this 
is also what gcc does.)

The way I planned to move forward was to separate VT_INT from VT_LONG, have 
VT_LONG be the new 64 bit register type on platforms that support it (and a 
synonym for VT_INT on 32 bit platforms), have VT_LLONG be treated as a 
synonym for VT_LONG on 64 bit targets (and treated via its current two 
register hoop jumping on 32 bit targets) so that 64 bit targets can actually 
be _simpler_ than the 32 bit ones because the two-register fandago drops out.

I also want to move some of the _internal_ stuff to some kind of big_enough_t 
type that gets typedefed to uint32_t or uint64_t as appropriate, depending on 
host+target.  You'll notice I chose to rewrite gen_opic() to do all constant 
propogation in terms of "long long", and then typecast back down to the 
smaller sizes.  Grischka instead chose to duplicate gen_opic() and have two 
versions.

This is one of the changes between my tree and mainline: I went for small and 
simple at the expense of speed in this case.  Grischka nominally chose speed 
over simplification, and duplicated a major codepath because of it, which I 
wouldn't want to maintain.  Notice that this is a fairly minor impact on 
speed anyway, partly because constant folding isn't a huge bottleneck but 
also because my way has one codepath it can keep in L1/L2 cache vs two 
codepaths competing for the cache space.  Also, it only has _any_ impact on 
speed on 32 bit platforms.  And I've been pondering some kind of ./configure 
option to support "long long" at all, which you can switch _off_ if you want 
the speed back and which can simplify the code in a lot of _other_ places 
too.  Plus that work has to be done anyway to properly support 64 bits 
if "long long" becomes a synonym for "long", so maintaining that config 
option becomes free...

I have lots of notes in my blog about what needs to be done to make the 
current x86_64 host actually _clean_ rather than "working by accident in 
places".  Some of the code generator stuff is treating int and pointer as 
interchangeable, which it is on 32 bit but not 64 bit.  I _think_ it's never 
taking an actual x86-64 pointer value and squashing it to a 32 bit int, but 
it is typecasting back and forth between 64 bit pointer variables and 32 bit 
int variables...

Fixing all this is a big job, sure.  But I've been thinking about it for over 
a year, and I have gotten the code to work pretty well on an x86-64 host 
already.  (That's what my laptop is, and that's what I mostly test.  I have a 
32 bit qemu image for 32 bit testing.  It's slower than messing with a 
chroot, but easier and cleaner for me to keep track of.)

> It's similar to the issue 
> I have with supporting soft float on ARM. I need doubles to be kept
> in two integer registers.

It should all be more cleanly factored out.  It's nice to have that 
capability, but it should be more generic if at all possible.

Half the work that needs to be done is code refactoring and cleanups.  I've 
done a lot of that in my tree, which is why I can't easily exhange patches 
with mainline anymore...

> > (Tried building running the x86 code generator on arm?
> > Segfault, unaligned access...)
>
> We could add a function/macro that reads unaligned ints.
> grep "(.*\* *)" *c should list all relevant (and some irrelevant) lines.

It's just a todo item.  I got lots of 'em. :)

> > Your version is still missing rather a lot of work I've done.  But I
> > can't change it, and nobody here wants to, so I'll stop bothering you
> > now.
>
> Even if there was someone who wants to change it, we can't because of
> the license. Can you change the license back to LGPL after having accepted
> patches from other people?

If I tracked exactly which patches those were, where they came from, and which 
license they were submitted under?  Yes, I can. :)

The reason I narrowed the license was because I the darn CVS tree not only 
wouldn't die, it started _following_ me, and I didn't want to encourage it.  
Preventing my code from being ported back to the otherwise largely defunct 
CVS tree was the _only_ reason for the license change, and I'd probably be 
happy to expand it back to LGPLv2 again if the CVS tree stopped being an 
issue.

Sure, I'd have to audit for third party patches, just to be sure, but that's 
maybe an hour's checking for me.  I recorded where everything came from.  
Keeping track of that sort of thing is basic code hygiene.

But the thing is, the CVS tree never stopped being issue.  And I'm tired of 
fighting it.

> > Daniel's work isn't based on my fork.  Nobody else's is.  Instead, what
> > work they do is based on the CVS version.
>
> I started working on tinycc before there was a fork

Everybody who is willing to deal with CVS started before I did my fork.  How 
many new developers has the project attracted in the past two years?

You think I'm the only person turned off by CVS, rather than just A) _vocal_ 
about it, B) willing to actually _do_ something about it (maintain my own 
darn fork in a real source control system)?

> and since patches
> for your version can not be used by Fabrice's version (because of the
> license, renaming, and a new directory structure), I'll stay here.

Exactly: the CVS version is "official", and even if it dies completely it's 
still "official".  I agree, I'm no longer fighting it.

However, calling the current CVS "Fabrice's version" is an interesting choice 
of words.  Fabrice has posted on this list exactly once this year, and 
exactly twice last year.  He put out a call for a new maintainer the year 
before that (back in 2006).  He has not been actively maintaining the project 
for around three years now.

The de-facto new maintainer is Grischka, who put together the 0.9.24 release 
(largely by porting over large chunks of my tree, although there were a 
number of other patches as well).  There hasn't been an official handoff that 
I'm aware of, and Grischka last posted to this list in may.  But the only 
real way to consider the CVS "Fabrice's version" is to imply that the project 
has been completely stagnant for years.

When I emailed Fabrice about maintainership back in 2006 and early 2007, he 
insisted that the repository must stay on Savannah (which only offered CVS at 
the time), and that changing off of CVS wouldn't fix anything.  That's the 
main reason I didn't bite: I would have been forced to maintain the project 
_in_cvs_, despite making it clear that was not an option for me.

After I complained about CVS enough that Fabrice told me to shut up about it, 
I stopped trying to change the main repository.  Instead I got Fabrice's 
permission to do my fork as a separate project (GPLv2 and all), and I 
probably put more effort into my fork over the past 2 years than every other 
tcc developer combined.  (Not because I'm all that good at it -- compilers 
are _not_ my area -- but because nobody else put in the time.)

But the thing is, that simply didn't matter.

> If this project switches away from CVS, I will as well, as will everyone
> else.

*shrug*  I _did_ switch away from CVS.  I can't control what anybody else 
does.

This thread is not about me trying to change the base project.  I stopped 
doing that when I was _asked_ to stop.  This thread is just me explaining why 
I've stopped working on tinycc.

This project has so far been quite happy to go down with the ship of CVS, and 
I'm no longer fighting it.  After two years of totally wasted effort I admit 
I'm slightly _bitter_, but I'm no longer throwing good development effort 
after bad.

> Btw., my last patches were based on 0.9.24, not on CVS.

I looked at 0.9.24 back when it came out:

  http://landley.net/notes-2008.html#17-03-2008

Didn't find anything new worth sucking into my fork at the time.  I've 
downloaded your patch tarball, and I'm vaguely curious about them. 
(Parentheses in the filenames is not a polite thing to do to Bash, but I can 
work around that if I try.)

But I'm no longer working _towards_ anything, so why bother?  The "official" 
archive is still  CVS, and my fork would never be anything other than a 
motivation for other people to fiddle with the CVS version.  I've lost 
interest in adding anything else to something I have to describe using the 
word "fork".

That's all I wanted to say, and I've said it.  It's your project now, not 
mine.  I have five other projects to work on, and if I really feel the need 
for a new compiler these days there's pcc and clang/llvm, both active 
projects that might be able to build a full Linux system with a finite amount 
of effort.

Tinycc was once closer to offering a serious open source alternative to gcc 
than anything else, and it's still simpler (and smaller, and faster), but 
part of the definition of its development community was a willingness to deal 
with development tools from the days of DOS.

>   Daniel

Rob

(P.S.  Judging by past history, my leaving will give the project another three 
months of indignant energy.  Maybe it'll be sustained this time, who knows?  
cc: me if you want me to see any replies, I'm unsubscribing from the list.)