Please enable JavaScript.
Coggle requires JavaScript to display documents.
PipeLib (Mail description (As we add more features, all our consumers need…
PipeLib
Mail description
-
The entire structure of the pipeline is defined in a library, developed and supported by the BARE team
This takes a large load off the developers, while giving the BARE team the ability to contribute pipeline feature from a central location.
Currently, every dev project has to copy a large amount of Kotlin code from another repository, and then maintain their own copy. Updates do not spread across the company, and when we want to enact central changes, such as introduce security auditing to builds, we need to manually many hundreds of changes (we currently have 680 pipelines).
-
With Pipelib, all a project needs is this:
-
No details, no curly braces, no worries! Just tell us what your project is called, and the library builds you this pipeline automatically:
As we add more features, all our consumers need to do is update the library version they use.
-
- Abstract TeamCity/Kotlin API away from developers
-
-
- Produce a test coverage report (even if developers did not configure it in their Gradle file)
- Store and render the coverage report in TeamCity
- Produce SonarQube quality reports
- Run SourceClear security scan
- If all tests pass, publish the built artifact on Nexus
- Create a pipeline for pre-merge tests, which is similar to the production pipeline, but does not publish artifacts.
The build team will create several types of pipelines, which will cover the entire structure and flow of the pipelines needed for different types of artifacts. Instead of copying and maintaining 500-1000 lines of Kotlin code, projects will only need to specify which type of pipeline they want to use.
The structure and steps that will comprise the pipeline will be maintained by the build team, who will be able to centrally track, manage, and modify the pipelines for the entire company, simplifying work for developers, and ensuring adherence to company standards and practices across all artifacts.
Today's pipelines consist of 500-100 lines of Kotlin, spread over 15 files that must be copied and maintained for each of our many pipelines. Most teams have multiple sets of Kotlin code to maintain separately. For comparison, a complete managed pipeline defintion will look like this:
Once you enable one of the managed pipelines for your repository, you will rarely need to modify. Any steps that need to be added will be added centrally by the build team
- Every pipeline is a unique creation, maintained by individual teams as TeamCity Kotlin code. Creating and maintaining the pipeline requires each team to ramp up on TC concepts and tools.
It is therefore expensive and intimidating for teams to create their own pipelines, and difficult for the build team to provide central services such as artifact packaging, auditing, KPI tracking, reporting, or adding mandatory security scans.
The question we attempted to answer in discovery is: "What common needs can we identify in the BTD processes of various teams, and how can we build common solutions for these use cases, so that creating new, fully automated build/test/deploy processes becomes trivial?"
Most code is written in Java, and most teams are comfortable writing short gradle scripts that build and test their code on a laptop
Most people expressed frustration with the complexity of TeamCity, the limits of its integration with BitBucket, and the effort required to maintain their pipelines
Most people felt that these are common needs, which would best be solved uniformly across the company.
Provide a common template for building BTD pipelines.
Hide most details of TeamCity, Bitbucket, and our other tools away from developers.
Make it trivial to create pipelines for new artifacts.
Enable the build team to add or modify build steps for all pipelines by making a change directly to the pipeline library. No steps necessary for individual pipelines to get the change.
-
We will create a library of TeamCity steps to encapsulate common steps that most artifacts go through
Using these prebuilt steps as foundation, we will create a library of complete, managed pipelines.
Managed pipelines allow developers to delegate the entire management of their pipelines to the build team.
-
By effectively outsourcing management of the pipelines, developers will save time and avoid the TeamCity learning curve.
By having fewer different pipelines to support, and only having to make changes once, build engineers will be better able to support engineering requests.