Output folder

Hey -

I've got Jenkins up and running for building and packaging.  It's very exciting. I can specify a tag from git and it automatically (with a bucket load of python as well admittedly) checks out the right source, cleans, builds, packages, signs and uploads carefully named installers to the server: Windows and Mac now!  I feel quite smug everytime I push the button.

However, I'm a bit nervous on the Mac.  I was going to use a separate user account for the builds but it gets its knickers in a twist about signing keys.  So it's building under my account. However this means that the build script is writing the plugins to the same folders as I'm using for my debugging and development. 

Is there a way of specifing the output folder without mucking with the script? I couldn't figure it out.  It looks like i"d need two introjucer build configurations set-up with different scripts in them to have alternate plugin output folders on the Mac. Which would leave room for a new dangerous inconsistency!

Options?

  • Ideal feature would be a command line option on introjucer to specify whether to place the .component .vst and .aaxplugin into their default locations or all into a single named folder.
  • Two post-build build scripts in introjucer, one for release mode and one for debug. 
  • Failing that maybe there's some way of using a single environment variable to control the build script... actually that might be the best option.

Any other suggestions?

I've recently been mulling over some ideas that would be an even more ambitious version of this.. The introjucer would actually be an ideal app for actually running a buildbot-style process that watches a git repo and automatically rebuilds + installs when it changes, because unlike a script or jenkins-style external app, it has access to all the internal project info like version numbers etc, and could easily invoke the compiler and run a script to do the builds. (This is definitely high on my to-do-list because it'd be extremely handy for us doing tracktion!)

Re: command-line option to copy the various plugin types to folders, that sounds like it could be really useful - if you have time to suggest some code, I'd be keen to help bash it into shape!

Well, it wasn't terribly hard to do it with Jenkins - aside from the bloody Apple and signing keys problem.  It might be better to just streamline the integration with that rather than add a new bunch of buildbot stuff to Introjucer?   You'd be able to make use of all the work and integration that's already been done with Jenkins then.  

Ref: plugin location. I'll have a peek at the script, I think it'd be a case of a tick-box and a define in the Introjucer code + an updated shell script. 

Broadly I think it's this bash horror story below but with the rest of the doXYZ functions in.  And then somehow set the flag and path from the environment.  I'll have a little play tomorrow



COPY_TO_OTHER_DESTINATION=0
OTHER_DESTINATION=~/BuildOutput/
# Functions take destination folder as parameter $1
function doAudioUnit() {
    AU=${1}${PRODUCT_NAME}.component
    if [ -d "$AU" ]; then 
        rm -r "$AU"
    fi
    cp -r "$original" "$AU"
    sed -i "" -e 's/TDMwPTul/BNDLPTul/g' "$AU/Contents/PkgInfo"
    sed -i "" -e 's/TDMw/BNDL/g' "$AU/Contents/$INFOPLIST_FILE"
    # Fix info.plist for AUs built with Xcode 3
    if [ -f "$DEVELOPER_DIR/Library/Developer/CoreAudio/AudioUnits/AUPublic/AUBase/AUPlugInDispatch.cpp" ]; then
        echo
    else
        echo "Removing AudioComponents entry from Info.plist because this is not a new-format AU"
        /usr/libexec/PlistBuddy -c "Delete AudioComponents" "$AU/Contents/Info.plist"
    fi
}
# try(this_format_flag, copy_function, install_target_path)
function try() {
    if [ $1 -gt 0 ]; then
        if [ $COPY_TO_OTHER_DESTINATION -gt 0 ]; then 
            $2 $OTHER_DESTINATION
        else 
            $2 $3
        fi
    fi    
}
try $copyAU doAudioUnit "~/Library/Audio/Plug-Ins/Components/"
try $copyVST doVST "~/Library/Audio/Plug-Ins/VST/"
try $copyVST3 doVST3 "~/Library/Audio/Plug-Ins/VST3/"
try $copyRTAS doRTAS "/Library/Application\ Support/Digidesign/Plug-Ins/"
if [ -d "/Applications/ProTools_3PDev/Plug-Ins" ]; then
    try $copyAAX doAAX "/Applications/ProTools_3PDev/Plug-Ins/"
if [ -d "/Library/Application Support/Avid/Audio/Plug-Ins" ]; then
    try $copyAAX doAAX "/Library/Application Support/Avid/Audio/Plug-Ins/

# This script takes the build product and copies it to the AU, VST, and RTAS folders, depending on 
# which plugin types you've built
original=$CONFIGURATION_BUILD_DIR/$FULL_PRODUCT_NAME

## These need to be set by some auto-magic from the Introjucer rather
## than being fixed here...
COPY_TO_OTHER_DESTINATION=1
INSTALL_TO_PLUGIN_FOLDERS=1
# Using ~ causes some problems with quotes around 
# filenames. And using no quotes causes problems 
# with filenames with spaces. So let's use ${HOME} 
# instead. 
OTHER_DESTINATION=${HOME}/TestBuildOutput/
# Create build output folder if we need to. 
test $COPY_TO_OTHER_DESTINATION -gt 0 && test ! -d "${OTHER_DESTINATION}" && mkdir 

$OTHER_DESTINATION
# These examine the binary to see what configurations have been built.
copyAU=`nm -g "$CONFIGURATION_BUILD_DIR/$EXECUTABLE_PATH" | grep -i 'AudioUnit' | wc -l`
copyVST=`nm -g "$CONFIGURATION_BUILD_DIR/$EXECUTABLE_PATH" | grep -i 'VSTPlugin' | wc -l`
copyVST3=`nm -g "$CONFIGURATION_BUILD_DIR/$EXECUTABLE_PATH" | grep -i 'GetPluginFactory' | wc -l`
copyRTAS=`nm -g "$CONFIGURATION_BUILD_DIR/$EXECUTABLE_PATH" | grep -i 'CProcess' | wc -l`
copyAAX=`nm -g "$CONFIGURATION_BUILD_DIR/$EXECUTABLE_PATH" | grep -i 'ACFStartup' | wc -l`
# Functions take destination folder as parameter $1
function doAudioUnit() {
    cp -r "$original" "${1}"
    sed -i "" -e 's/TDMwPTul/BNDLPTul/g' "${1}/Contents/PkgInfo"
    sed -i "" -e 's/TDMw/BNDL/g' "${1}/Contents/$INFOPLIST_FILE"
    # Fix info.plist for AUs built with Xcode 3
    if [ -f "$DEVELOPER_DIR/Library/Developer/CoreAudio/AudioUnits/AUPublic/AUBase/AUPlugInDispatch.cpp" ]; then
        echo
    else
        echo "Removing AudioComponents entry from Info.plist because this is not a new-format AU"
        /usr/libexec/PlistBuddy -c "Delete AudioComponents" "${1}/Contents/Info.plist"
    fi
}
function doVST() {
    cp -r "$original" "${1}"
    sed -i "" -e 's/TDMwPTul/BNDLPTul/g' "${1}/Contents/PkgInfo"
    sed -i "" -e 's/TDMw/BNDL/g' "${1}/Contents/$INFOPLIST_FILE"
}
function doVST3 {
    cp -r "$original" "${1}"
    sed -i "" -e 's/TDMwPTul/BNDLPTul/g' "${1}/Contents/PkgInfo"
    sed -i "" -e 's/TDMw/BNDL/g' "${1}/Contents/$INFOPLIST_FILE"
}
function doRTAS {
    cp -r "$original" "${1}"
}
function doAAX {
    cp -r "$original" "${1}"
}
function execute() {
    echo Installing $1 to $2
    if [ -d "${2}" ] ; then 
        echo "Removing previous version: ${2}"
        rm -r "${2}"
    fi
    do${1} "${2}" # do{$1}!? - this really is the bastard child of eval
}
# try(this_format_flag, 2:copy_function, 3:install_target_path, 4:extension)
function try() {
    if [ $1 -gt 0 ]; then
        if [ $COPY_TO_OTHER_DESTINATION -gt 0 ]; then 
            DST="${OTHER_DESTINATION}${PRODUCT_NAME}${4}"
            execute ${2} "${DST}"
        fi 
        if [ $INSTALL_TO_PLUGIN_FOLDERS -gt 0 ]; then       
            DST=${3}${PRODUCT_NAME}${4}
            execute ${2} "${DST}"
        fi
    else 
        echo "Skipping: $2 (binary is not built with $2 compatibility)"
    fi    
}
PROTOOLS_DEV_FOLDER="/Applications/ProTools_3PDev/Plug-Ins/"
PROTOOLS_STD_FOLDER="/Library/Application Support/Avid/Audio/Plug-Ins/"
USER_PLUGINS_FOLDER="${HOME}/Library/Audio/Plug-Ins/"
################################################################################################
#   FLAG        FUNCTION    TARGET FOLDER                                           EXTENSION
try $copyAU     AudioUnit   "${USER_PLUGINS_FOLDER}Components/"                     .component
try $copyVST    VST         "${USER_PLUGINS_FOLDER}VST/"                            .vst
try $copyVST3   VST3        "${USER_PLUGINS_FOLDER}VST3/"                           .vst3
try $copyRTAS   RTAS        "/Library/Application\ Support/Digidesign/Plug-Ins/"    .rtas
if [ -d "${PROTOOLS_DEV_FOLDER}" ]; then
try $copyAAX    AAX         "${PROTOOLS_DEV_FOLDER}"                                .aaxplugin
fi
if [ -d "${PROTOOLS_STD_FOLDER}" ]; then
try $copyAAX    AAX         "${PROTOOLS_STD_FOLDER}"                                .aaxplugin
fi
################################################################################################
    

This is what I was after.  It's got two options for installation, and you can choose both at once.  

Install to a single folder or install to user folders. 

 

I haven't tried jenkins, but have witnessed the scripting carnage involved in setting up a buildbot system (and that was only for OSX, not even cross-platform!), which made me realise that it'd be a darned sight easier just to write the same logic in C++ in the introjucer, and leave that running on a build machine.

And how does cross-platform scripting work on systems like Jenkins? For something like tracktion, we need to do a win32 build, and the idea of involving cygwin would just be ghastly! The great thing about doing it inside the introjucer is that it already has huge amounts of code to understand the different paths involved in building the project on different OSes, so there wouldn't need to be separate scripts for each target platform.

Well, I dropped cygwin out of the process when I moved to Jenkins. Prior to that I was using bash scripting for the build. 

Now I've got it with a cross-platform python script that takes it from a checked out source to having the installer on the server.  It delivers:

  • Version extraction and tagging. 
  • Calling introjucer to resave.
  • Calling the compiler
  • Copying files
  • Packaging
  • Signing
  • Basic santity checks.
  • Uploading to the server

Jenkins wraps that with the following functionality: 

  • GIT checkout
  • Reporting
  • Logging
  • GUI
  • Scheduling
  • Keeping a library of the things there are to build
  • Master/slave configuration

And the whole setup enforces a clean build and reproducable build environment.

But there are some other reasons why it might be good to use Jenkins rather than a custom solution: 

  • Plugins: https://wiki.jenkins-ci.org/display/JENKINS/Cppcheck+Plugin https://wiki.jenkins-ci.org/display/JENKINS/Warnings+Plugin
  • Support for all the other repsistories
  • User accounts 
  • And so on...

But maybe with pithy new features introjuced to the introjucer it might be a taster beverage straight out of the carton ... as it were.

I'll send you the python over. Maybe that's the bit that's the key bit? It's a bit plugin/ISS/Packages specific tho.