Externalizing fingerprint storage for Jenkins is a Google Summer of Code 2020 project.
We are working on building a pluggable storage engine for fingerprints (see JEP-226).
File fingerprinting is a way to track which version of a file is being used by a job/build, making dependency tracking easy. The fingerprint engine of Jenkins can track usages of artifacts, credentials, files, etc. within the system. Currently, it does this by maintaining a local XML-based database which leads to dependence on the physical disk of the Jenkins master.
Allowing fingerprint storage to be moved to external storages decreases the dependence of Jenkins instances on the physical disk space and also allows for tracking the flow of fingerprints across instances of Jenkins connected to the same external storage.
Advantages of using external storage drivers:
Remove dependence on Jenkins master disk storage
Can configure pay-as-you-use cloud storages
Easy Backup Management
Better Reliability and Availability
Fingerprints can be tracked across Jenkins instances
Along with this API, we are also working on a reference implementation in the form of a plugin, powered by Redis.
As phase 1 of this project comes to an end, this blog post serves as a summary of the progress we made to the entire Jenkins community.
With PR-4731, we introduce a new fingerprint storage API, allowing configuring custom storage engines.
We exposed the following methods in the new
Saves the given Fingerprint in the storage.
Fingerprint load(String id)
Returns the Fingerprint with the given unique ID. The unique ID for a fingerprint is defined by Fingerprint#getHashString().
void delete(String id)
Deletes the Fingerprint with the given unique ID.
Returns true if there is some data in the fingerprint database corresponding to the particular Jenkins instance.
Redis Fingerprint Storage Plugin uses the new External Fingerprint Storage API to store the fingerprints in a Redis instance.
The alpha release (version
0.1-alpha-1) for the plugin was drafted, and can be installed using the experimental update center.
Follow along the following steps after running Jenkins to download and install the plugin:
Configure the Update Site URL as: https://updates.jenkins.io/experimental/update-center.json
Submit, and then press the
Redis Fingerprint Storage Pluginand check the box along it.
Install without restart
The plugin should now be installed on your system.
Once the plugin has been installed, you can configure the Redis server details by following the steps below:
Scroll to the section
Redis Fingerprint Storage Configurationand fill in the required details:
Host– Enter hostname where Redis is running
Port– Specify the port on which Redis is running
SSL– Click if SSL is enabled
Database– Redis supports integer indexed databases, which can be specified here.
Connection Timeout– Set the connection timeout duration in milliseconds.
Socked Timeout– Set the socket timeout duration in milliseconds.
Credentials– Configure authentication using username and password to the Redis instance.
Enabled– Check this to enable the plugin (Note: This is likely to be removed very soon, and will be enabled by default.)
Test Redis Connectionto verify that the details are correct and Jenkins is able to connect to the Redis instance.
Now, all the fingerprints produced by this Jenkins instance should be saved in the configured Redis server!
Some of the topics we aim to tackle in the next phases include extending the API, fingerprint cleanup, migrations (internal→external, external→internal, external→external), tracing, ORM, implementing the saveable listener, etc.
Originally posted on Jenkins Blog