Monday, July 30, 2012

Gmake Acrobatics

Lets start with obvious. Suppose you have in your Makefile several variables:

DB_PORT := 5432
DB_USER := joe
DB_NAME := test

Variable values come from some configuration file outside of this Makefile. There is no point of replication such information & holding it in 2 places (config & Makefile). So you start thinking like this: "I'll just read my config file from my Makefile and assign variables dynamically. That's easy."

Suppose the config is in json format. Using handy jsontool we can write:

  DB_HOST := $(shell json < myconfig.json)  

Okay. But with this approach we need to execute jsontool every time for each variable. For n variable this will be exactly n forks. Suddenly every task in your Makefile becomes a little (or not a little) sluggish.

It is possible, of course, to execute jsontool only once and get a newline separated 'list' of all values:

$ json db.port db.user < myconfig.json

But how do you map those into Makefile variables?

In every other language this would be very simple: iterate over a list of variable names; for each name, construct a string var := value and feed it to eval function. Fot example, in lovely CoffeeScript:

values = ['a', 'b']
MyEval "#{v} := #{values[count++]}" for v,count in ['DB_HOST', 'DB_PORT']  

Try to translate this into gmake code & you will struggle badly. gmake can iterate on a string, splitting it with spaces. It has eval function which can parse & evaluate its makefile language. It even has some simple helper functions for manipulating strings, for example $(word 2,this is nice) would return the word 'is'.

What is doesn't have is basic arithmetic support. You can't add 1 + 1 in it without executing a shell script or whatever.

Googling may bring up this link to you, which is hilarious in its very own way but has a great idea: if we need a simple counter, we can start it with an empty string (yes, string), call gmake function $(words $(string)) which will return 0. Then we concatenate the string with another string containing a space & a char (for example, ' x'), call function $(words $(string)) again & it will return 1. And so on.

This is what I ended with:

# Create a batch of variables on-the-fly.
# _left contains variable names, _right--their corresponding values.
_right := $(shell json db_name db.user db.port < myconfig.json)
_n := 1
define _dvars
$(i) := $$(word $$(words $$(_n)),$$(_right))
_n := $$(_n) x
$(foreach i,$(_left),$(eval $(_dvars)))

It works but looks ugly. Those define...endif construction is just a way to define a variable that contains newlines. eval function evaluates it twice for every iteration. This is why we need $$ in front of every dynamic construction except of $(i) parameter.

Hint: user Rake & don't waste your time.