Wednesday, October 28, 2009

SparkBuild, GNU make and Tcl

Eric Melski из Электрической Тучи, вчера написал об утилите, которая подменяет собою gmake и генерирует (в процессе сборки) XML файл, в котором можно потом обнаружить всякую интересную статистику.

Утилита называется SparkBuild и она бесплатна. Не open source, но ее можно свободно использовать для коммерческих проектов.

SparkBuild состоит из 2-х главных компонентов:

  • emake, который (как обещают) и есть drop-in замена GNU Make.
  • sbinsight, который парсит output из emake и рисует графики, таблички и проч.

emake написана на C++ и Boost; sbinsight--это Tcl/Tk 8.5+ (симпатичные виджеты из 8.5 выдают себя моментально). Как используется Тикль в sbinsight--вопрос интересный. Судя по наличию tclKitInit и Mk4tcl в потрохах скомпилированного sbinsight, это разновидность Starpack'а.

В качестве теста, я попробовал собрать Texinfo 4.13 с помощью SparkBuild. Хотя последний распространяется только для Windows и лайнукса, у меня все чудесно заработало на FreeBSD 7.2 и:

% pkg_info | grep ^linux
linux-f8-expat-2.0.1_1 Linux/i386 binary port of Expat XML-parsing library (Linux
linux-f8-fontconfig-2.4.2_1 An XML-based font configuration API for X Windows (Linux Fe
linux-f8-xorg-libs-7.3_3 Xorg libraries (Linux Fedora 8)
linux_base-f8-8_11  Base set of packages needed in Linux mode (for i386/amd64)


% pwd
% make configure

% cd work/texinfo-4.13

Теперь наступает самое интересное. Вместо пускания gmake, набираем:

% emake --emake-annodetail=basic,waiting,env | tee anno.xml

И вместо привычного make output, на stdout вылазит гора XML'а, и после конца сборки у нас появляется файл:

% ls -l anno.xml
-rw-r--r--  1 alex  wheel  367670 Oct 28 15:22 anno.xml

Желаю вам успеха в его чтении глазами. В поставке SparkBuild есть Tcl-утилитка anno2log, которая выковыривает из anno.xml то, что писал бы на stdout gmake. То есть, в теории, можно пустить emake вот так:

% emake --emake-annodetail=basic,waiting,env | tee anno.xml | anno2log

На практике, текущая версия anno2log буферизирует stdout, поэтому результат видно только по окончанию сборки.

Чтобы полюбоваться на статистику (то, ради чего), печатаем:

% sbinsight anno.xml &
24 KB

Информация о конкретной target:

7 KB

График симуляции сборки texinfo на кластере:

3 KB

Кластерная сборка, естественно и к несчастью, в SparkBuild не входит, потому что она есть главный продукт Электрической Тучи имени Джона Остераута.

Wednesday, October 14, 2009

Quotes from Coders at Work

4 of 15 interview are good, 8 are brilliant, 1 is totally useless and 2 are boring.

Person Rating
Jamie Zawinski 10
Brad Fitzpatrick 10
Douglas Crockford 9
Brendan Eich 9
Joshua Bloch 8
Joe Armstrong 9
Simon Peyton Jones 5
Peter Norvig 10
Guy Steele 10
Dan Ingalls 6
L Peter Deutsch 10
Ken Thompson 10
Fran Allen 2
Bernie Cosell 10
Donald Knuth 10

Jamie Zawinski

So I write a new [Emacs byte] compiler and Stallman's response is, "I see no need for this change." And I'm like, "What are you talking about? It generates way faster code." Then his next response is, "Okay, uh, send me a diff and explain each line you changed." "Well, I didn't do that--I rewrote it because the old one was crap." That was not OK. The only reason it ever got folded in was because I released it and thousands of people started using it and they loved it and they nagged him for two years and finally he put it in because he was tired of being nagged about it.


There was actually a point, early on in Netscape, where part of our build process involved running "emacs -batch" to manipulate some file. No one really appreciated that.

Brad Fitzpatrick

Before I had any official job, I got some hosting account. I got kicked off of AOL for writing bots, flooding their chat rooms, and just being annoying. [...] I also wrote a bot to flood their online form to send you a CD. I used every variation of my name, because I didn't want their duplicate suppression to only send me one CD, because they had those 100 free hours, or 5,000 free hours. I submitted this form a couple thousand times and for a week or so the postman would be coming with bundles of CDs wrapped up.

My mom was like, "Damn it, Brad, you're going to get in trouble." I was like, "Eh their fucking fault, right?" Then one day I get a phone call and I actually picked up the phone, which I normally didn't, and it was someone from AOL. They were just screaming at me. "Stop sending us all these form submissions!" I'm not normally this quick and clever, but I just yelled back, "Why are you sending me all this crap? Every day the postman comes! He's dropping off all these CDs!" They're like, "We're so sorry, sir. It won't happen again." Then I used all those and I decorated my dorm room in college with them. I actually still have them in a box in the garage.

Douglas Crockford

I've managed projects where we're up against a deadline and we had people saying, "Yeah, I'm almost done," and then you get the code, and there's nothing there, or it's crap, or whatever, and they're nowhere close to done. In management, those are the experiences you hate the most and I think code reading is the best way of not getting trapped like that.

[...] it [code reading] requires a lot of trust on the part of the team members so there have to be clear rules as to what's in bounds and what's not. If you had a dysfunctional team, you don't want to be doing this, because they'll tear themselves apart. And if you have a dysfunctional team and you're not aware of it, this will reveal it pretty quickly. There's a lot that you can learn, a lot that's revealed by this process. It feels unnatural at first, although once you get into the rhythm of it, it feels extremely natural.

Brendan Eich

I'm secure enough to think I could go do something that was a fine sky castle for myself, but I'm realist enough to know that it would be only for myself and probably not fine for other people. And what's the point? "If I'm only for myself", you know, Hillel the elder, "what am I?" I am not JavaScript. In the early days, it was such a rush job and it was buggy and then there was some Usenet post Jamie Zawinski forwarded me. He said, "They're calling your baby ugly." I have real kids now; I don't have to worry about that.

Joe Armstrong

I think the lack of reusability comes in object-oriented languages, not in functional languages. Because the problem with object-oriented languages is they've got all this implicit environment that they carry around with them. You wanted a banana but what you got was a gorilla holding the banana and the entire jungle.


Are you driven by the problems or by the solutions? I tend to favor the people who say, "I've got this really interesting problem." Then you ask, "What was the most fun project you ever wrote; show me the code for this stuff. How would you solve this problem?" I'm not so hung up on what they know about language X or Y. From what I've seen of programmers, they're either good at all languages or good at none. The guy who s a good C programmer will be good at Erlang--it's an incredibly good predictor.

Peter Norvig

They [NASA] aren't software guys. They say, "Software is this necessary evil. Straight line code, I can sort of understand; if it's got a loop in it, that's kind of iffy. Then if there's a branch statement inside the loop, ooooh, that's getting away from what I can solve with a differential equation in control theory." So they're distrustful.

L Peter Deutsch

So basically that's what I did [Ghostscript]--data structures first. Rough division into modules. My belief is still, if you get the data structures and their invariants right, most of the code will just kind of write itself.


Seibel: I think Larry Wall described it [Python] as a bowl of oatmeal with fingernail clippings in it.

Deutsch: Well, my description of Perl is something that looks like it came out of the wrong end of a dog. I think Larry Wall has a lot of nerve talking about language design--Perl is an abomination as a language.

Ken Thompson

I've never been a lover of existing code. Code by itself almost rots and it's gotta be rewritten. Even when nothing has changed, for some reason it rots.


Seibel: I know Google has a policy where every new employee has to get checked out on languages before they're allowed to check code in. Which means you had to get checked out on C.

Thompson: Yeah, I haven't been.

Seibel: You haven't been! You're not allowed to check in code?

Thompson: I'm not allowed to check in code, no.

Seibel: You just haven't gotten around to it, or you have philosophical objections to the Google coding standard?

Thompson: I just haven't done it. I've so far found no need to.


I would try out the language as it was being developed and make comments on it. It was part of the work atmosphere there [AT&T]. And you'd write something and then the next day it [C++] wouldn't work because the language changed. It was very unstable for a very long period of time. At some point I said, no, no more.

In an interview I said exactly that, that I didn't use it just because it wouldn't stay still for two days in a row. When Stroustrup read the interview he came screaming into my room about how I was undermining him and what I said mattered and I said it was a bad language. I never said it was a bad language. On and on and on. Since then I kind of avoid that kind of stuff.

Bernie Cosell

Programmers are the worst optimizers in the world. They always optimize the part of the code that's most interesting to optimize, and almost never get the part of the code that actually needs optimization. So you get these little nuts of very difficult code that have no point.

Donald Knuth

Like in the parts that I'm writing now, I'm starting out with stuff that's in math journals that is written in jargon that I wouldn't expect very many programmers to ever learn, and I'm trying to dejargonize it to the point where I can at least understand it. I try to give the key ideas and I try to simplify them the best I can, but then what happens is every five pages of my book is somebody's career.


The first rule of writing is to understand your audience--the better you know your reader the better you can write, of course. The second rule, for technical writing, is say everything twice in complementary ways so that the person who's reading it has a chance to put the ideas into his or her brain in ways that reinforce each other.


The problem is that coding isn't fun if all you can do is call things out of a library, if you can't write the library yourself. If the job of coding is just to be finding the right combination of parameters, that does fairly obvious things, then who'd want to go into that as a career?


We already talked about literate programming--that's a radical departure, that I'm viewing myself as an expositor rather than trying to just put together the right instructions. Dijkstra came out with that same evolution. In the end his programs were even more literate than mine in the sense that they didn't even go into the machine. They were only literate.


I always try things that are at my limit. If I had to go back and write those kinds of programs again, the easier ones, I wouldn't make so many mistakes. But now that I know some more, I'm trying to write harder stuff. So I make mistakes because I'm always operating at my limit. If I only stay in comfortable territory all the time, that's not so much fun.


So inevitably we're going to have bugs unless we decide we're never going to write anything that stretches our capabilities.

Sunday, October 4, 2009

time value too large/small to represent

После перезагрузки виртуальной машины, в жерле которой мирно крутятся некоторые Tcl CGI скрипты, эти самые скрипты внезапно перестали работать, выплевывая вот такое сообщение:

time value too large/small to represent
    while executing
"::tcl::clock::ConvertLocalToUTC $date[set date {}]  $TZData($timeZone)  $changeover"
    (procedure "::tcl::clock::scanproc'%A, %d %B %Y, %T'c" line 33)

Это была реакция на самую невинную комманду:

clock scan {Monday, 28 September 2009, 22:11:00} -format {%A, %d %B %Y, %T}

Конечно, она замечательно исполняется, если ее попробывать, например в tkcon. В же чем проблема?

Как оказалось, environment variable TZ при старте Apache (еще до login prompt в FreeBSD) не успевает быть установленной--то есть Apache ее inherit, если она есть. А для этого нужно:

# setenv TZ Europe/Kiev
# /usr/local/etc/rc.d/apache22 restart

И тогда все начинает опять работать правильно. Капитан Очевидность потирает руки.

Или, можно, на всякий случай, всегда устанавливать TZ руками в Tcl-скрипте:

set env(TZ) Europe/Kiev

Или, сделать вот такой симлинк:

# ln -s /usr/share/zoneinfo/Europe/Kiev /etc/localtime

Последний способ, по идее, есть самый универсальный.