RE: Complex/Large Lilypond projects and build automation

[Jim Duke]

David,

 

Thanks for your response.  I’m looking into using Gradle – and I think I’ll be happiest if I invest the time to develop one or more Gradle plugins to define the notion of a Lilypond project.  I have a steep learning curve in front of me since I’m not familiar with Gradle.  So part of my interest in using Gradle here is so I have an excuse to learn Gradle.

Gradle uses a convention-over-configuration approach (their words), and so that leads to the question: what conventions to people use in file system layout, use of templates, include files, etc.  The Gradle plugin that I write will assume a certain conventional layout – so I’ll start with what I’m using.  But I’m interested in what other people are doing.  Also, as part of the plugin I’ll eventually make the attempt to discover the outputs from a particular Lilypond file.  That’s a longer term goal, and very much non trivial.  So I’ll probably start with the user being required to explicitly define the inputs and outputs.  I already need to do that with Make – so not too big a deal for me.

 

Are you able to share any Gradle build files?  I would love to see what you did as I’m learning Gradle.

 

Thanks again,

 

Jim

 

From: David F. <address@hidden>
Sent: Saturday, December 7, 2019 6:17 PM
To: Jim Duke <address@hidden>
Cc: Lilypond User (address@hidden) <address@hidden>
Subject: Re: Complex/Large Lilypond projects and build automation

 

 

On Dec 7, 2019, at 2:35 PM, Jim Duke <address@hidden> wrote:

I have a large Lilypond project for several hymnals.  The project is organized into a subdirectory structure with each hymn in a separate directory.  For each hymn I produce several products: an 8.5x11 PDF, a 6x9 PDF, a set of images formatted for projection at 4x3 and 16x9 aspect ratios, and a set of part dominant MP3’s for use in learning the hymns.  These products are then assembled at a higher level into a comprehensive PDF of the hymnal including front material and index; and a package of the slide images organized to be compatible with a Slide production product used at my church for assembling a song service; as well as assembling metadata files to be uploaded into a website I maintain that provides access to these hymns and provides simple search capabilities and internet access to these products.

 

I use Make (that old workhorse) to automate the process; but that has some distinct limitations.  So I was wondering what tools others may be using to aid them in building larger Lilypond projects.  What, if anything, are you using?  How well does that work for you?

 

Jim, I’m right behind you!  I have a collection of just over 100 hymns that are mostly Spanish language, but about 20 of those also have bilingual versions.  My build system creates proof PDFs, 4x3 and 16x9 slide images and 4x3 and 16x9 PowerPoint slide decks which get synchronized to a shared Dropbox folder.  I started off with make, but switched to gradle because I was already using that at work.  I’ve got about 400 lines of Kotlin/gradle code.  I haven’t built any hymnals yet, but I plan to.

 

So far I feel like I’ve gotten further with gradle than I could have with make, but lately I’ve found gradle to be more and more annoying.  Both make and gradle struggle with 1-to-many and (especially) many-to-1 type build tasks.  So, for example, taking 8 png files and combining them into one Powerpoint file is doable in gradle, but you’re really fighting the build system and incremental builds start to fail in certain cases (like when a song goes from fitting on 6 slides to taking up 8 slides).  There are other annoyances.

 

Just a couple of weeks ago, I came across this blog post:

  "The only build system that might someday replace make”

  https://apenwarr.ca/log/20101214

 

So I intend to dig in to the design of the redo system and see if it can match the functionality provided by gradle without the pain and annoyances.

 

David