Many to Many joins with Ecto and Elixir

Ecto is an DSL for writing queries and interacting with databases in Elixir.  It has a style similar to ActiveRecord, but since there aren’t objects, you can’t really call it an ORM.

Like ActiveRecord, it has has_many and belongs_to, but it doesn’t have has_many :through yet. Here is how I implemented a DAG using a many-to-many join in Ecto anyways

The basic idea is I have a Nodes table.  And each Node can have multiple parents and multiple children.  The most straightforward implementation of this is to have a many-to-many join table and then use that with INNER JOIN to query for parents or children.

First, I needed to create the tables using Ecto’s migration feature.

Then I define the models using the Ecto model DSL.

This wasn’t quite enough though. I ran into trouble getting Node to preload its children and parents via NodeToNode. I ended up adding help functions to the Node module.

This now lets me write an association.

This works as far as I can tell with my unittests. I’m not completely comfortable with Ecto’s association proxy, so there might be a better way to do this that is more idiomatic. I hope that has_many :through support is added soon so I can concentrate on my business logic instead of mundane joins. I think more documentation of reading and writing associations is necessary.

If you know a better way to express this, please let me know.

named tasks in leader_cron

I’ve been using erlcron to run scheduled tasks, but since each node in an erlang cluster would have its own copy, it didn’t help with having the tasks run in only one location per-cluster. Then I found leader_cron, which uses gen_leader to elect a single node in the cluster to run the tasks.

This is great stuff, but it doesn’t solve the problem of which node should schedule the task. For this purpose, I added the feature of named tasks, so that if you add a task with the same name, you receive a {error, already_exists} response. This way all the nodes can try to schedule the tasks, but only one will succeed.

I’ve submitted it to the maintainer and I hope he likes it, but it seems useful enough to hold onto even if he has reservations.

Agile and Hyperproductivity

I was wondering around my office, which is housed in a co-working location, when I saw on a whiteboard in another company’s office, “Agile: Hyperproductive!” As an engineer type, I had to restrain myself from going in and telling them they were wrong, but I do have some thoughts on the subject.

I like Scrum. I think in the future for most teams, Scrum and similar processes won’t be controversial. They’ll just be how software engineers do things. That said, it’s not going to make you more productive. Its sprints have a higher overhead with daily standups, retrospectives, planning, and grooming meetings. The smaller the sprint, the higher the overhead is.

I’ve seen claims that Scrum will increase productivity by 800! That is incredible. And unbelievable. Are people typing faster? Are they designing faster? My speed of implementing is the same whether for waterfall or Scrum.

With the additional overhead of more meetings, you’re arguably *less* productive compared to an ideal waterfall scenario. But that’s okay, because a development cycle that doesn’t run into issues is a myth. Either a design flaw was found, or the business requirements changed, or some key player leaves. Something will happen, and Scrum’s smaller cycles deals with that better than a waterfall process.

I truly believe the team will come out ahead, although the sweet-spot of the best sprint length is different for all groups. I’ve worked with sprints from weekly to tri-weekly in size and it really depends on the culture of the project.

Just don’t say Scrum makes you more productive.

A nice, hype-free look at Scrum meetings.

Require.js and HAML

I’ve been having very good luck with requirehaml. I had been looking around at various templates, but none of them seemed to support compiling before delivery to the client. This script does that and compiles them into easily verifiable javascript files, packaged up to be included with my existing require.js AMD build.

So far, I haven’t actually had that much HTML to include, but hopefully that will change as my project takes shape and grows.

Now I just need to find something to do the same for SASS.

AMD modules and web developers

Now that require.js has reached 1.0, it’s probably time to start shouting its praises from every rooftop so that javascript library authors include support for it. Require.js is an implementation of the AMD draft specification which adds simple module support for javascript in the browser. This means no more namespace pollution (if you use it properly).

Unfortunately, we’re talking about web developers who don’t seem to understand basic principals like avoiding namespace pollution. I mean, how can you even have a conversation with someone with such a misconception of good practices?

Jeremy Ashkenas, the creator of CoffeeScript, Backbone, and Underscore also seems to have misconceptions about the purpose of AMD.

I was really hoping that someone with such a worthy track record would have seen the light by now. Luckily the require.js guy himself is pushing for AMD support in Underscore with a new pull request. Once that is pulled in, it looks like Backbone is next on the list.

Since the next version of jQuery will have AMD support, I imagine that the rest of the libraries will follow since they seem to look up to jQuery for best practices.

And then, one part of the nightmare that is working with javascript will be gone.

How to setup a pretty good dev environment on the mac

(This was a quick write up for a coworker, but others might find it useful)

This assumes you have XCode. I mean, really now.

First thing to do is install mac homebrew. It’s a package manager of useful open source software. You install it by pasting into the terminal:

/usr/bin/ruby -e "$(curl -fsSL https://raw.github.com/gist/323731)"

then just to make sure it’s working, run:

brew install -v git

Next we want to install the node package manager.

It’s a simple

curl http://npmjs.org/install.sh | sh

Lastly, I like to setup rvm, for managing multiple ruby installations and keeping my base machine clean of stray gems.

bash < <(curl -s https://rvm.beginrescueend.com/install/rvm)

RVM is a little more complicated since you also have to add a little something to your shell file so that the paths are loaded correctly

echo '[[ -s "$HOME/.rvm/scripts/rvm" ]] && . "$HOME/.rvm/scripts/rvm"

That's *about* it. I'm still looking into how to setup the RVM equivalent for python, but it seems that community is still figuring out their tools and a clear solution hasn't emerged yet.

New books!

I bought a couple books today. I picked up Managing Humans and Real World Haskell.

Managing Humans really needed a better editor. Maybe I’m not in the manager mindframe enough to understand it, but it jerked around and contradicted itself.

I’m hoping the Haskell book isn’t as disappointing. I’ve been wanting to understand functional programming for a long time, but never got around to picking up a book that explained how to model real programs that rely on input from the outside world.

What the hell, autoconf?

There’s a new autoconf out, autoconf-2.63.  It continues autoconf’s fine tradition of having the suckiest release schedule and quality of the GNU toolkit.  You would think that they would increment the major number whenever they release a non-backwards compatible release, but they don’t for shear perversity.  I guess 2.63 just rolls off the tongue, and I can’t blame them.  The ridiculousness of having to test for different versions of your platform configuration tool boggles me.  I’m at a loss for words at how much the autoconf maintainers have failed their core mission.

I’m not even going to go into their choice of m4 as the file format.  That probably made sense at the time. But that the thing is implemented in sh is just stupid.  

Really, what the autoconf maintainers should do is what the make maintainers never had the guts to do.  Realize that their software is making the world a worse place, pick a worthy successor, and never update their project again.  In my perfect world, autoconf is allowed to commit hari-kari.  Meanwhile, automake is lined up against a wall and shot at dawn.

On the plus side, this might be the chance for everyone to finally move to a sane build tool like waf or cmake.

there but for the grace of god…

I’m so glad I’m not an Apple Release Engineer today. Four major releases today. iPhone 3G, iPhone Firmware 2.0, MobileMe, and iTunes 7.7.

It’s a huge task to synchronize all those releases and it eats up company resources like crazy. You have to have enough people to focus on each task separately.

MobileMe still isn’t working right. They’ve been extending the maintenance downtime further and further.

They have my sympathies.