“Much that once was is lost. For none now live who remember it.”Lord of The Rings
The quot above describes really well the current state in my old team, and oddly enough the “Ring” fits very well with the solution we had for our build requirements.
As a Tech. Lead for a professional services team I found myself looking at countless projects and git repositories of the same type as we provide more and more solutions to our customers. It was always another WCF service or a REST net core service or a DLL.
Regardless of the type we always package them the same. We publish the WCF\ REST or build on release the DLL zip the entire content and ship it out to our deliver server for other reams to grab and deploy
The process above was done Manually (Barbaric!!!) before I joined the team, so I decided to change this.
The goal was to create a reusable job template that will help us to create new Jenkins jobs for every solution we had, so I immediately dove into Jenkins Pipelines while they are awesome when working on small amount of project after a while we hit a snag with them because of the following couple of reasons:
- Every solution had its own pipeline in his git repository, this caused dozens of Jenkins jobs been created and it was very hard to manage them all
- When a change was required during the build process or publication (like server change) we had to god and update all pipelines (I know we could have use a variable but we didn’t)
So one day I was playing around with bash and figured that I could write a publish scrip to rule them all. and here we are, the magic script (s) that make our build process work with a single parametrize job in Jenkins with the following input params
- GIT_URL – the git repo url to build
- BRANCH – the git branch to build
- SOLUTION_TYPE – .net core \ wcf \ dll (closed list)
- SOLUTION_FILE_NAME – (OPTIONAL!) the sln file that should be used for build \ publish only if its different than the git repository name
The first script Jenkins will run is BuildVars.sh its goal is to establish some global variables the will be used in other scripts.
# Remove echo
set +x
echo .
echo .
echo === Parse Solution ID from Git URL ===
basename=$(basename ${GIT_URL})
echo $basename
RepoName=${basename%.*}
echo $RepoName
echo .
echo .
echo === Cloning Repository ${GIT_URL} ===
git clone ${GIT_URL} -b ${BRANCH}
cd ${RepoName}
echo .
echo .
echo === Setting Default Build Variables ===
# This is the way we identify the version of the releas
VERSION_TAG=$(git describe --always --dirty --long --tags)
VERSION_TAG=${VERSION_TAG/-dirty/}
VERSION_TAG=${VERSION_TAG//-/.}
# Check if SOLUTION_FILE_NAME is provided and overwrite SOLUTION_ID
if test -z "$SOLUTION_FILE_NAME"
then
echo SOLUTION_FILE_NAME param was not provided, assuming that solution file is set to ${RepoName}
export SOLUTION_ID=${RepoName}
else
echo Setting SOLUTION_ID to provided ${SOLUTION_FILE_NAME}
export SOLUTION_ID=${SOLUTION_FILE_NAME}
fi
export SOLUTION_FILENAME="${SOLUTION_ID}.sln"
export PUBLISH_PROJECT_TARGET_DIR=${SOLUTION_ID}
export PUBLISH_PROJECT_TARGET_FILENAME="${SOLUTION_ID}.csproj"
export PUBLISH_DIR_LOCAL="/d/DEV/Releases/${SOLUTION_ID}"
export NUGET_REPO_GLOBAL="https://nuget.org/api/v2/"
export NUGET_REPO_PS="http://localhost:8090/nuget"
echo SOLUTION_ID = ${SOLUTION_ID}
echo VERSION_TAG = ${VERSION_TAG}
echo SOLUTION_FILENAME = ${SOLUTION_FILENAME}
echo PUBLISH_PROJECT_TARGET_DIR = ${PUBLISH_PROJECT_TARGET_DIR}
echo PUBLISH_PROJECT_TARGET_FILENAME = ${PUBLISH_PROJECT_TARGET_FILENAME}
echo PUBLISH_DIR_LOCAL = ${PUBLISH_DIR_LOCAL}
echo NUGET_REPO_GLOBAL = ${NUGET_REPO_GLOBAL}
echo NUGET_REPO_PS = ${NUGET_REPO_PS}
echo
echo
echo === Overwriting Build Variables from build.props.ini ===
if [ ! -f build.props.ini ]; then
echo "build.props.ini file not found skipping overwrites!"
else
config=$(cat build.props.ini)
echo Loaded following overwrites
echo $config
echo $pwd
source ./build.props.ini
fi
echo
echo
echo === Final Build Variables ===
echo SOLUTION_ID = ${SOLUTION_ID}
echo VERSION_TAG = ${VERSION_TAG}
echo SOLUTION_FILENAME = ${SOLUTION_FILENAME}
echo PUBLISH_PROJECT_TARGET_DIR = ${PUBLISH_PROJECT_TARGET_DIR}
echo PUBLISH_PROJECT_TARGET_FILENAME = ${PUBLISH_PROJECT_TARGET_FILENAME}
echo PUBLISH_DIR_LOCAL = ${PUBLISH_DIR_LOCAL}
echo NUGET_REPO_GLOBAL = ${NUGET_REPO_GLOBAL}
echo NUGET_REPO_PS = ${NUGET_REPO_PS}
cat > ${WORKSPACE}/generated.jenkins.vars <<EOL
RepoName=${RepoName}
SOLUTION_ID=${SOLUTION_ID}
VERSION_TAG=${VERSION_TAG}
SOLUTION_FILENAME=${SOLUTION_FILENAME}
PUBLISH_PROJECT_TARGET_DIR=${PUBLISH_PROJECT_TARGET_DIR}
PUBLISH_PROJECT_TARGET_FILENAME=${PUBLISH_PROJECT_TARGET_FILENAME}
PUBLISH_DIR_LOCAL=${PUBLISH_DIR_LOCAL}
NUGET_REPO_GLOBAL=${NUGET_REPO_GLOBAL}
NUGET_REPO_PS=${NUGET_REPO_PS}
...
EOL
echo
echo
So there is a lot going on there let me talk about the highlights.
Versioning
We use git describe --always --dirty --long --tags to get the latest tag with the commit count since, this proved out to be a good strategy specially when you have bunch of different version running around the then a bug comes in for one of them, go figure which code base you need to check without it.
There is a different process we do to patch versions just before the build is done but this is a whole different story to tell.
Solution ID
Mostly all our solutions have the same name for both .sln file and the git repository. so we use basename to get the repository name form the git URL
Variable Overrides
I wanted to allow any project to overwrite any variable to give the flexibility to change anything if required per project so we use source ./build.props.ini to load a build.props.ini file that will contain any overrides required for the variables.
From this point we just use the input variable SOLUTION_TYPE to identify how we should publish and run the appropriate script
This is how it looks in the Jenkins UI
But this was not enough! I don’t want to copy paste the git URL select the project type or event navigate to Jenkins when i start a build, so I needed some way to trigger it automatically from multiple different git repositories.
Introducing Url based Job trigger:
Now we are getting there! but still I need a way to allow our developers to trigger this request in a simple manner. Presenting: ps-release.sh script this script will search the current directory and will try to find out what is the solution type, what is the remote git repository url and issue the POST request to the jenkins server
#!/bin/bash
TOKEN='YourPasswordHere'
GIT_URL='Unknown'
bitbucket_remote_host='git@bitbucket.org:your_company/'
github_remote_host='git@github.com:your_company/'
fullOriginUrl=$(git config --get remote.origin.url)
echo fullOriginUrl $fullOriginUrl
fullOriginUrl=${fullOriginUrl/\/src/}
echo fullOriginUrl2 $fullOriginUrl
gitRepoName=$(basename $fullOriginUrl .git)
echo gitRepoName $fullOriginUrl
echo "Trying to determine git or bitbucket remote:[$fullOriginUrl]"
echo
shopt -s nocasematch
if [[ $fullOriginUrl =~ "github" ]]; then
echo 'github'
GIT_URL="$github_remote_host$gitRepoName.git"
elif [[ $fullOriginUrl =~ "bitbucket" ]]; then
echo 'bitbucket'
GIT_URL="$bitbucket_remote_host$gitRepoName.git"
else
echo 'Cannot determine git remote source its not git nor bitbucket.'
exit 125
fi
echo "Trying to determine release project type"
projectFile=$(find -name "*.csproj" | head -n 1)
svcFile=$(find -name "*.svc" | head -n 1)
outputType=$(grep -oPm1 "(?<=<OutputType>)[^<]+" $projectFile)
TargetFramework=$(grep -oPm1 "(?<=<TargetFramework>)[^<]+" $projectFile)
echo "Project File: ${projectFile}"
echo "SVC File: ${hasSvcFile}"
echo "Output Type: ${outputType}"
echo "TargetFramework: ${TargetFramework}"
if [[ $svcFile != "" ]]; then
echo "Found svc file in solution, assuming WCF project."
SOLUTION_TYPE='WCF'
elif [[ $outputType =~ "Library" ]]; then
echo "Found OutputType=Library in .csproj file in solution and not svc files, assuming DLL project."
SOLUTION_TYPE='DLL'
else
echo "Could not identify WCF or DLL project, assuming .Net Core"
SOLUTION_TYPE='NET_CORE'
fi
SOLUTION_FILE_NAME=$(basename $(find -name "*.sln") .sln)
BRANCH=${1:-master}
echo
echo
echo "Variables are set to:"
echo "GIT_URL: [$GIT_URL]"
echo "SOLUTION_FILE_NAME: [$SOLUTION_FILE_NAME]"
echo "BRANCH: [$BRANCH]"
echo "SOLUTION_TYPE: [$SOLUTION_TYPE]"
requestUrl="http://jenkins:8080/view/PS/job/PS.Release/buildWithParameters?token=${TOKEN}&GIT_URL=${GIT_URL}&BRANCH=${BRANCH}&SOLUTION_TYPE=${SOLUTION_TYPE}&SOLUTION_FILE_NAME=${SOLUTION_FILE_NAME}"
echo "calling url:[$requestUrl]"
result=$(curl --request GET \
--max-time 5 \
--write-out %{http_code} \
--silent \
--output /dev/null \
--url $requestUrl_external_ip)
echo result is: $result
So this will work well.. whats left for the DEV is install under /usr/bin/ run “sh ps-release.sh” and presto the build starts!
But wait there is more… I love command line and this is very nice for me, most of our team are windows guys and gals so I wanted something more familiar to them, I wanted this to become a button in the context menu of the folder I’m in or the folder i have clicked. Presenting: Windows Registry hack for adding right-click context menu Items
There is nothing like a good Windows Registry hack in the morning! here is the .reg script and here is the ps-release-context-menu.reg
Windows Registry Editor Version 5.00
[HKEY_CLASSES_ROOT\Directory\shell\ps_release]
@="PS-Release"
"Icon"="C:\\WINDOWS\\system32\\Defrag.exe"
[HKEY_CLASSES_ROOT\Directory\shell\ps_release]\command]
@="cmd.exe /s /k pushd \"%V\" && sh ps-release.sh"
[HKEY_CLASSES_ROOT\Directory\Background\shell\ps_release]
@="PS-Release"
"Icon"="C:\\WINDOWS\\system32\\Defrag.exe"
[HKEY_CLASSES_ROOT\Directory\Background\shell\ps_release\command]
@="cmd.exe /s /k pushd \"%V\" && sh ps-release.sh"
Code Skunk 