aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2022-10-12ansible/hosts: add build2-deb11build-ansibleosmith/wip-ci-packagingOliver Smith1-0/+1
Add it to the config. I've used this to enable ipv6 in docker, so libosmocore builds don't fail in the socket test when building rpm/deb packages on that host in docker. Related: OS#2385 Change-Id: Ibe8e2a638fcbc7c0a69c211385a870b1a82b6fef
2022-10-12jobs/gerrit-verifications: set pipeline concurrentOliver Smith1-0/+1
Allow running multiple gerrit-verification pipelines of the same project at the same time to use the jenkins nodes more efficiently. Note that the concurrent setting of the build jobs are not changed. Some projects have this set to false, and for those, there will be no more than one build job running at the same time. The build job gets started from the pipeline, along with other jobs like lint and deb/rpm package building. Related: OS#2385 Change-Id: Ie2a19013e73d96a937e8f83c815c6a82cfe0768d
2022-10-12jobs/gerrit-verifications: link to new wiki pageOliver Smith1-0/+3
Related: OS#2385 Change-Id: Ia4eac0c39fd156db1fc30e86ef55c48c6105912f
2022-10-12jobs/gerrit-verifications: add osmo-ciOliver Smith1-0/+4
Run the linter again on osmo-ci. Related: OS#2385 Change-Id: Ie6d8a99647846fb4008953b444bba642cbcf3b1e
2022-10-12jobs/gerrit-verifications*: drop unsupported argsOliver Smith2-4/+0
Fix warnings when updating the job with Jenkins Job Builder: WARNING:jenkins_jobs.modules.triggers:Gerrit Trigger property 'no-name-and-email' is deprecated WARNING:jenkins_jobs.modules.triggers:Gerrit Trigger property 'trigger-for-unreviewed-patches' is not supported in this plugin version Related: OS#2385 Change-Id: I6a0057738955bc2507cb5d8c4d48edf652c3529f
2022-10-12jobs/gerrit-verifications: add approvals commentOliver Smith1-0/+10
Related: OS#2385 Change-Id: I4dfc0f84ffe584523bbff4c41295968a07ba097d
2022-10-12jenkins-gerrit: tweak notify settingOliver Smith1-2/+7
Instead of notifying everybody listed as reviewers and in CC: * notify the owner if the build failed * notify nobody if the build was successful Related: OS#2385 Change-Id: I7c6c183b98624eb75d5dccd9766ee3ff5568b06e
2022-10-12jenkins-gerrit: add 0-9 to re_start_buildOliver Smith1-1/+1
Let it match "Starting building: gerrit-osmo-ttcn3-hacks-build #5". It failed on the 3 in ttcn3 before. Related: OS#2385 Change-Id: I247af55e2c0e3a2dd088ab1c951d8535cfc93229
2022-10-12jenkins-gerrit: meaningful re_start_build errorOliver Smith1-0/+4
Related: OS#2385 Change-Id: Ib29b7ce69a27453b460f946ad67bd12faffc2c89
2022-10-11obs: support Osmocom projects without prefixOliver Smith3-7/+12
Allow running build_srcpkg.py with "osmo_dia2gsup" instead of "erlang/osmo_dia2gsup", because that's how it gets passed along from jobs/gerrit-verifications.yml for the binary package build verifications. Related: OS#2385 Change-Id: I09304b219e7002495bd3fcce61bc68c34d5ffcd8
2022-10-11jobs/gerrit-verifications: write summary commentOliver Smith4-1/+280
Instead of having multiple mails from jenkins per submitted gerrit patch, only have one at the end of the pipeline that writes a summary with all relevant links and votes +1/-1 for verified. This also resolves the problem we had for a long time, that the link for the build job didn't directly point to the relevant console output. Instead it pointed to a matrix job that didn't have a direct link to the relevant job. The summary looks like this: 1 failed: [build] https://jenkins.osmocom.org/jenkins/job/gerrit-osmo-bsc-nat-build/a1=default,a2=default,a3=default,a4=default,label=osmocom-gerrit-debian9/11/consoleFull 3 passed: [rpm] https://jenkins.osmocom.org/jenkins/job/gerrit-binpkgs-rpm/5/consoleFull [deb] https://jenkins.osmocom.org/jenkins/job/gerrit-binpkgs-deb/5/consoleFull [lint] https://jenkins.osmocom.org/jenkins/job/gerrit-lint/11/consoleFull Build Failed Related: OS#2385 Change-Id: Idcab969e1b5ca4e0f1383bee8f36f2d1aac4f624
2022-10-11obs: config: update note about testing packagesOliver Smith1-3/+2
Related: OS#2385 Change-Id: If6d772052ca64ef93536bf8729a72271f67fbecb
2022-10-11obs/README: add overview with build_binpkg.pyOliver Smith1-0/+19
Related: OS#2385 Change-Id: I1b741ad325e6541e6d59dfd50de66bee805e56fe
2022-10-11jobs/gerrit-binpkgs: new job for deb, rpm packagesOliver Smith2-9/+205
Build deb and rpm packages for each patch submitted to gerrit for the projects in the list. Example: https://gerrit.osmocom.org/c/osmo-bsc-nat/+/29492 Related: OS#2385 Change-Id: I7ca8869c2e9f2e7c74a360933be12b5c2b47c2fc
2022-10-11obs: build_binpkg: use almalinux:8 namingOliver Smith3-7/+4
Instead of using "--docker centos:8" as argument and translating it later on in the code to use "almalinux:8" as base distribution, use "--docker almalinux:8" as argument. It was brought up in code review that this makes it less confusing. Related: OS#2385 Change-Id: Id8298e8bafe065010f7bc00f1ff261aa6431ed4c
2022-10-07jobs/gerrit-lint: run as part of pipelineOliver Smith2-116/+65
Reduce mails by running gerrit-lint as part of the gerrit-verifications pipeline. Related: OS#2385 Change-Id: I5f1340248cd116c5fa5134eee3fae50baf7ef851
2022-10-07jobs/gerrit-verifications: add param to skip buildOliver Smith1-0/+10
Prepare to unify the list of projects in jobs/gerrit-verifications.yml and jobs/gerrit-lint.yml by allowing to skip the build for some repositories (e.g. docker-playground is linted but we don't run a contrib/jenkins.sh there). Change-Id: Ie6264d44120798c0ecbcaf9c63d9b95f74ae9e37
2022-10-07jobs/gerrit-verifications: wrap in pipeline jobOliver Smith1-24/+113
Prepare to reduce the amount of comments sent to gerrit from jenkins and resulting mails. Instead of triggering the jobs directly from gerrit, trigger a pipeline job that calls the original job (now called gerrit-{repos}-build) and will in future patches call all other gerrit verification jobs too (linter, building deb/rpm packages). Change-Id: Ica208a5177739655cdd493d461da146d440cd542
2022-10-06obs: build_binpkg: optimize osmo-gsm-manuals-devOliver Smith4-4/+57
Installing osmo-gsm-manuals-dev plus depends takes a long time. Don't do this for every build, instead do it once when building a second docker container and then use that. Related: OS#2385 Change-Id: I8475bd954352b572197795ad4cd9461e39896d48
2022-10-06obs: build_binpkg: debian: don't install manpagesOliver Smith1-0/+5
Configure dpkg to not extract man pages. Otherwise it will spend some time regenerating the man page index whenever installing build dependencies before starting to build a package. Related: OS#2385 Change-Id: I1c9e3883b976e023c96dfd59eb147770f7ad99a7
2022-10-06obs: add build_binpkg.pyOliver Smith9-3/+275
Add a script to build deb/rpm packages, as it would be done on obs.osmocom.org. This will be used by jenkins to verify deb and rpm builds for each submitted gerrit patch. I have attempted to use 'osc build' instead of directly calling apt-get build-dep and dpkg-buildpackage (and rpm equivalents). Using 'osc build' would have the advantage that the build works as close to the OBS build as possible. However it would try to install dependencies with sudo, so we would need to have sudo available in the docker container that builds the untrusted code from gerrit. Let's not do that. Related: OS#2385 Change-Id: I4c6b5d61af35df98cbc70d9ddc8ad36d38a9ce18
2022-10-06obs: lib.docker: fix redundant get_image_name()Oliver Smith1-1/+1
Related: OS#2385 Change-Id: Ifa4e316db63c1a3bb9e0d845e7f755d04cf1297e
2022-10-06obs: run_in_docker_and_exit: no os.path.basenameOliver Smith3-6/+5
Don't use os.path.basename on the script_path parameter passed to the function. A future patch will pass a script that is inside the data subdirectory to this function, therefore this is needed. Related: OS#2385 Change-Id: Ide78d976f9af445c4c8d8748bc274d7289064769
2022-10-06obs: prepare for multiple dockerfilesOliver Smith3-7/+27
Rename Dockerfile to build_srcpkg.Dockerfile and adjust related code to allow using a different Dockerfile for building the binary packages in a future patch. Related: OS#2385 Change-Id: I8ef7944a4a81acd6c915998f37139eebad2b2d3e
2022-10-06obs: run_in_docker_and_exit: remove args paramOliver Smith3-5/+5
No need to pass args here. It was only used for args.verbose, and that isn't necessary since we don't need to restore the verbose mode. Related: OS#2385 Change-Id: I620cdef46e18f4c66644f14003caf2183c89686f
2022-10-06obs: run_in_docker_and_exit: --rm on extra lineOliver Smith1-1/+2
Put it on an extra line like the other arguments to prevent it from getting overlooked. Related: OS#2385 Change-Id: Ie7dea2734e7c47766dfe64b7091bc4f5f72d53a5
2022-10-06jobs/osmocom-obs: add job for master feedOliver Smith3-1/+11
Add one new job for building source packages and sending them to obs.osmocom.org. Trigger it from all master-* jobs. I've also considered adding one job per existing master job that would only update one package at a time (master-libosmocore-obs, master-osmo-bsc-obs, ...). With some additional development effort it should be possible, and it would make each individual master OBS job faster. But given that with the current implementation it only takes 20s to 30s for *all* packages if there are no changes, as it compares git remote HEAD with the version currently on OBS before starting to clone repositories and building the source packages (similar to Osmocom_OBS_latest_obs.osmocom.org), it didn't seem worth optimizing. Set concurrent to false as the triggers from master-builds will likely cause it to run multiple times in parallel otherwise. Related: https://jenkins.osmocom.org/jenkins/view/OBS/job/Osmocom_OBS_master_obs.osmocom.org/ Related: https://obs.osmocom.org/project/show/osmocom:master Related: OS#2385 Change-Id: I53a494f13f81ae837f2d362c54e1bdf13f121db3
2022-10-06obs: update_obs_project: add master feedOliver Smith6-10/+58
Add a new master feed, where packages are updated as soon as patches get merged to master. Upload a commit_$COMMIT.txt file for each package in this feed and use it to determine if the package needs to be updated or not. In most packages the commit is already part of the version in the dsc file, e.g. "libosmocore_1.7.0.38.c3b90.dsc", but that's harder to parse and is more likely to have a hash collision (just 5 characters). Related: OS#2385 Change-Id: I3b0b4f4876b8c1eeb61f20d903a6f2cac6e99638
2022-10-06obs: build_srcpkg_if_needed: switch if branchesOliver Smith1-3/+5
Make it slightly easier to read, before extending the latest branch with master too in a follow-up patch. Related: OS#2385 Change-Id: I4f1d4e250eb4d8163f1ded29106a44deb0d0feff
2022-10-06obs: lib.git.get_default_branch: add functionOliver Smith1-3/+6
Refactor checkout_default_branch so the get_default_branch logic can be used from another function in a future patch. Related: OS#2385 Change-Id: If3f0ab500ff165d81f61b19ad41af90bd39dce8f
2022-10-04coverity: fix prepare_source_Osmocom.sh terminating earlyVadim Yanitskiy1-1/+1
Since [1] was merged this script is executed with '-xe' making it fail immediately if a command returns a non-zero code. grep may return 1 if pattern is not found, so let's suspend it. Change-Id: I21a01478b64015fd699ed31634bf2fb1dc8f6750 Fixes: [1] Icf6546271a01cd9a3dab1ce0d1bc02807d08ad85
2022-10-01scripts: osmo_git_clone_url(): properly handle 'satellite/osmo-gmr'Vadim Yanitskiy1-0/+3
Coverity job currently fails because it fails to fetch osmo-gmr.git. Change-Id: I4d52dbf0f81332dc1856944a6a3c7376cb1e0638 Fixes: I17590d0b0b46274e2ceb007e20d7d1dc039f4e8d
2022-10-01coverity/prepare_source_Osmocom.sh: execute with '-xe'Vadim Yanitskiy1-1/+1
Change-Id: Icf6546271a01cd9a3dab1ce0d1bc02807d08ad85
2022-09-26Replace most remaining git.osmocom.org urlsOliver Smith15-18/+18
Replace the legacy git.osmocom.org clone URLs with either gerrit or gitea in most places. Besides not using legacy urls anymore, the advantage of using gerrit directly is that we don't have a delay of 10 min to several hours on the repository. Not replaced: * ansible/roles/osmocom-workstation: not sure if still used, would need a bit of a rewrite to use logic similar to osmo_git_clone_url(). Related: https://osmocom.org/projects/cellular-infrastructure/wiki/Git_infrastructure Change-Id: I90409c9e7f4db4e1765647021625f9acfccaa78b
2022-09-26scripts/common: remove osmo_git_clone_dateOliver Smith1-10/+0
This had been added for debugging purposes and isn't used anymore. Also one less hit for git grep git.osmocom.org. Change-Id: I10cbb598f0d5d675034ca87568a8b6f1920f2ad3
2022-09-26coverity: use osmo_git_clone_urlOliver Smith3-1/+4
Stop cloning from git.osmocom.org by using the shared function. Change-Id: I17590d0b0b46274e2ceb007e20d7d1dc039f4e8d
2022-09-26coverity/prepare_source_Osmocom: remove test -dOliver Smith1-1/+1
This was redundant, mkdir -p doesn't error if the directory structure exists. Change-Id: Ic6253ecb9bfd5f8b2d744988b8825ef90e2e6d5e
2022-09-26osmo_git_clone_url: git.osmocom.org -> giteaOliver Smith1-5/+10
Update the function to not use legacy URLs anymore and allow adding future gitea URLs more easily. Update the description of the function to reflect the new motivation for using this. Change-Id: Ibf53ca2c444fae82af73e417410faa5b16f5de59
2022-09-26Revert "scripts/obs: make shallow git clones"Oliver Smith1-1/+1
I didn't realize during code review that the git-version-gen script we use in Osmocom repositories need the git history to generate a proper version string. For example (libosmocore): 1.7.0.38-c3b90.202209240002 With a shallow clone, it is: 1.7.0.202209250002 Revert the shallow git clone patch to get the proper version strings again. This reverts commit 115262d8412d420fb4a2ff26da6002b75a7a4ade. Related: https://jenkins.osmocom.org/jenkins/job/Osmocom_OBS_nightly_obs.osmocom.org/102/console Related: https://jenkins.osmocom.org/jenkins/job/Osmocom_OBS_nightly_obs.osmocom.org/103/console Change-Id: I6176e52187d12d264eee82e238051ca65e0c8f4d
2022-09-23scripts/obs: make shallow git clonesMax1-1/+1
While generating srcpkg we're not really interested in entire project history (which can be quite heavy) - let's only get the bare minimum from git. Change-Id: I71629c244d2c29728a88871e83b20493341321c8
2022-09-22lint: accept BE spelling of 'acknowledgement'Oliver Smith1-1/+0
Related: https://gerrit.osmocom.org/c/libosmo-gprs/+/29402 Change-Id: Ia1a08295a7c7ed6a77f0055d66a161423d8f17f0
2022-09-22jobs: enable ttcn3-upf-testNeels Hofmeyr1-1/+0
Related: I7a651bb85861de4b2677a731b9f17fa0d790c621 (docker-playground) Change-Id: I5964979b3db86268f6011df0afbee7131557f405
2022-09-21build_srcpkg.py: add option to use git-reviewMax5-3/+17
Having an option to explicitly checkout sources for package from gerrit using numeric ID comes in handy for testing changes. Related: OS#4107 Change-Id: I403c2083ef8c3f6f966c5b376fb13ea46d8a7a2b
2022-09-21scripts/obs/*.py: add option to ignore required programs checkMax3-2/+9
Not all the required programs are actually required to build subset of packages. Add option to ignore the default check. Change-Id: I8b5eaedab87b586fb1ca4c22746be4dfbb3c2176
2022-09-21obs: lib.run_cmd: set stdin to subprocess.DEVNULLOliver Smith1-3/+3
Don't pass stdin to the programs, as we expect them to run non-interactively and also don't show the program's output unless -v is used or the exit code is not 0. Change-Id: I7e893101c2a3e7b005659ec72aa44fa932b7ccd9
2022-09-21obs: update_obs_project: check if oscrc existsOliver Smith3-1/+14
Check if the oscrc exists, as otherwise the osc command will interactively prompt for the credentials. If running without verbose mode (the default), the command output is hidden and so it would just appear to be hanging without this patch if the config doesn't exist. Change-Id: I6f140b2c1d171d3019fd259399e6a46da9c6451f
2022-09-21obs: don't print verbose output twice on errorOliver Smith1-1/+6
When running in verbose mode, it's enough that the output is displayed while the command is running. Don't print it again on error. Change-Id: I4fcf2421a6f82bd94ba098b39f58e5d3f18aa8fd
2022-09-21scripts/obs/*.py: log name of function calling commandMax1-1/+3
Change-Id: If4ccc0eb44da4ea743f3858b09c762ee6a138f1d
2022-09-15lint: ignore MACRO_WITH_FLOW_CONTROLOliver Smith1-0/+2
It seems that we don't care about this one, e.g. here: https://gerrit.osmocom.org/c/osmo-msc/+/28848 Change-Id: I79da5a426db59031e3b16aecedeaa1498c91e847
2022-09-14Revert "jobs: add ttcn3-hnbgw-test-pfcp"Neels Hofmeyr1-3/+0
Instead merging into ttcn3-hnbgw-test/ This reverts commit cbdbd7b204c076d210755eaaafe874125b21dcd0. Change-Id: I296a33d7db85a90bce328250d75cedc2fa04633f
generated by cgit v1.2.3 (git 2.25.1) at 2024-09-21 07:00:27 +0000