summaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorNeels Hofmeyr <neels@hofmeyr.de>2017-10-27 22:10:17 +0200
committerNeels Hofmeyr <neels@hofmeyr.de>2017-10-27 22:37:22 +0200
commitf42a1cfa501d67a0fa96def4a1ff352c2b214620 (patch)
tree93cc91fc530be98aaf53e034edf342dc981a611f
parent26f9d8768b644ae0dd20b0e671b949a340c954e4 (diff)
add osmo-clean-workspace.sh, use in osmo-deps.sh
So far, each jenkins job does its own cleanup, more or less well. Also, jenkins git config offers the 'Clean before checkout' option, which seems to fail when there are non-writable leftovers from a failed 'make distcheck'. Furthermore, our jenkins build slaves have unused compiled binaries piling up by the gigabytes: each matrix build x each parallel build and each compiled dependency therein builds .o, .a, .so and executables plus installs them to a local prefix, and just leaves them sitting around to rot until the job runs again. Instead, we want to clean them out when building is done. All of this calls for a unified cleanup script that knows how to clean a workspace properly, to run once before and once after each jenkins build. Here it is. Use that function in osmo-deps.sh instead of duplicating cleanup steps. Change-Id: I2409b2928b4d7ebbd6c005097d4ad7337307dd93
-rwxr-xr-xscripts/osmo-build-dep.sh1
-rwxr-xr-xscripts/osmo-clean-workspace.sh42
-rwxr-xr-xscripts/osmo-deps.sh6
3 files changed, 48 insertions, 1 deletions
diff --git a/scripts/osmo-build-dep.sh b/scripts/osmo-build-dep.sh
index 2a107ed..8104ba7 100755
--- a/scripts/osmo-build-dep.sh
+++ b/scripts/osmo-build-dep.sh
@@ -40,7 +40,6 @@ set -x
mkdir -p "$deps"
cd "$deps"
-rm -rf "$project"
osmo-deps.sh "$project"
cd "$project"
if [ -n "$branch" ]; then
diff --git a/scripts/osmo-clean-workspace.sh b/scripts/osmo-clean-workspace.sh
new file mode 100755
index 0000000..11a5b72
--- /dev/null
+++ b/scripts/osmo-clean-workspace.sh
@@ -0,0 +1,42 @@
+#!/bin/sh
+# Clean workspace.
+# This should be the first and last step of every jenkins build:
+# a) to make sure the workspace has no build artifacts from previous runs.
+# b) to reduce disk space lost to unused binaries; parallel and/or matrix
+# builds create numerous workspaces, blowing up disk usage.
+# Note that if a build fails, the last steps will not run, hence calling this
+# as last step cleans only in case there was no build failure, i.e. where we
+# don't need to keep anything anyway.
+#
+# Assume $PWD is a git clone's root dir. Usually, that's also the jenkins
+# workspace root. Do not wipe subdir 'layer1-headers' as well as all dirs under
+# '$deps'. These are assumed to be git clones that do not need to be re-cloned
+# every time. Do a 'git clean' in each of them individually. If '$deps' is not
+# defined or the mentioned dirs do not exist, nothing special happens, so this
+# script can be safely run in any git clone in deeper subdirs of the workspace.
+
+set -ex
+
+# make sure no write protected cruft is in the way. A failed 'make distcheck'
+# has a habit of leaving a non-writable dir tree behind.
+chmod -R +w .
+
+# wipe all local modifications
+git checkout -f HEAD
+
+# wipe all unversioned leftovers, except deps gits.
+git clean -dxf -e "$deps" -e "layer1-headers"
+
+# leave the deps checkouts around, to not clone entire git history every time,
+# but clean each git of build artifacts.
+if [ -d "$deps" ]; then
+ for dep_dir in "$deps"/* ; do
+ git checkout -f HEAD
+ git -C "$dep_dir" clean -dxf
+ done
+fi
+
+if [ -d "layer1-headers" ]; then
+ git checkout -f HEAD
+ git -C "layer1-headers" clean -dxf
+fi
diff --git a/scripts/osmo-deps.sh b/scripts/osmo-deps.sh
index 86da9a6..c256a6a 100755
--- a/scripts/osmo-deps.sh
+++ b/scripts/osmo-deps.sh
@@ -8,5 +8,11 @@ fi
cd $1
git fetch origin
+
+# Cleanup should already have happened during a global osmo-clean-workspace.sh,
+# but in case the caller did not (want to) call that, let's also do cleanup in
+# the dep subdir separately:
+osmo-clean-workspace.sh
+
git reset --hard origin/master
git rev-parse HEAD