Ansible and IBM Community Grid

The world is currently a very different place than it was only a few months ago and we have come up with some ideas on how we can help our community in dealing with this new reality. The Ansible team has started a “Here to Help” webinar series where myself and other Ansible engineers are spending time with smaller groups of people to try and help them with technical challenges: https://www.ansible.com/here-to-help-webinar-series. The goal of these webinars is strictly to help! Regardless of if folks are only using open source technologies and not Red Hat products, we want to use this time to help them solve automation challenges, and help us brainstorm use-cases that can help others.

Another idea we recently implemented is integrating IBM’s World Community Grid into our workshops. World Community Grid enables anyone with a Linux, Windows or Mac computer (or an Android smartphone for some projects)  to donate their unused computing power to advance scientific research on topics related to health and sustainability. In fact, one of their projects is specifically going to help combat COVID-19. This blog post will cover what our workshops are and how we can use idle CPU time to help IBM’s World Community Grid. We use Ansible to install, start and register the BOINC client and integrate it into all Red Hat Enterprise Linux (RHEL) instances used in Ansible Automation Workshops.

 

Some background on workshops

We offer free Ansible Automation Workshops that spin up various topologies for students to learn the basics of Ansible Automation. Each topology can have up to four RHEL machines spun up in various public clouds. We have thousands of these Linux instances spun up all across the world for various workshops including Singapore, Japan, the United States and Germany. In 2019, we had over 25,000 topologies spun up which means we had over 50,000 RHEL Linux machines up during the year.

Often, a lot of these instances are sitting idle waiting for students to start accessing them. Red Hat Solution Architects might spin up the entire topology hours before they actually teach a workshop to verify they have everything setup correctly. These instances can also remain up for hours after the class finishes because students have requested additional time to have access to the lab environment. All of these instances are dedicated instances so we are paying for them regardless of how they are used (versus CPU time). We use dedicated instances (versus something like spot instances) because we need to guarantee that the instructor and students can use them at any time during a 48 hour period.

 

What is World Community Grid?

World Community Grid has a noble purpose and describes itself as an effort to create the world’s largest public computing grid to tackle scientific research projects that benefit humanity. Specifically, my team was inspired by this comment on their blog post (linked above).

One easy way to help join the fight against COVID-19 is through IBM’s World Community Grid, which crowdsources the surplus computing power that volunteers like you and me have, and uses it for scientific research. World Community Grid is being deployed in the fight against COVID-19, soon launching an important project from Scripps Research called “OpenPandemics – COVID-19.”

Essentially, you install a software agent onto a computer. Large CPU workloads are divided up amongst the broader community (thousands of people’s personal PCs, servers, etc) and run these calculations during CPU idle times (where the computer is not being used). The agent, or piece of software running on the computer is BOINC which is distributed under the LGPL open source license by University of California, Berkeley.

The setup is simple:

• Create an account on https://www.worldcommunitygrid.org/
• Choose which projects you want to contribute to (machines are auto-registered to a default profile that you choose when you create an account)
• Download the software to your respective type of machine (Windows, MacOS, Linux, etc)
• Let it run and watch your statistics on their website

 

How does Ansible fit in?

The workshops use Ansible to provision, configure and manage instances on public clouds. I simply can add a new Ansible Role to our workshop provisioner to enable the EPEL repository, install the boinc-client and register it to my newly created account. The manual installation directions can be found on Berkeley’s website here. Our Ansible Automation Workshop project is also open source and can be found here https://github.com/ansible/workshops.

For Ansible Automation Workshops we are using RHEL 8, so if you are using different distributions you would have to adjust accordingly. Here is the simple four task role:

---
- name: Install EPEL
  dnf:
    name: "https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm"
    state: present

- name: install boinc-client with package module
  package:
    name:
      - boinc-client
    state: present

- name: enable and start boinc-client
  systemd:
    name: boinc-client
    state: started
    enabled: yes

- name: World Community Grid registration
  command:
    cmd: "boinccmd --project_attach {{ boinc_url }} {{ boinc_auth }}"
    chdir: /var/lib/boinc/
    creates: "/var/lib/boinc/account_{{ boinc_url }}.xml"

1. dnf module – This module installs, upgrades, removes and lists packages and groups with the dnf package manager. DNF is the next-generation version of the Yellowdog Updater, Modified (yum), a package manager for .rpm-based distributions. In order to grab boinc-client, I need to have the EPEL repo enabled for RHEL 8.
2. package module – This module Installs, upgrades and removes packages using the underlying OS package manager. For this task, I install the boinc-client.
3. systemd module – Controls systemd services on remote hosts. For this task, I make sure boinc-client is enabled (Whether the service should start on boot) and it is started
4. command module – For the last module, there is not a dedicated BOINC Ansible Module (at least not yet!) so I can simply use the generic command module to register the machine. The boinc_url is “www.worldcommunitygrid.org” while the boinc_auth can be found by logging into https://worldcommunitygrid.org, clicking on Settings and navigating to My Profile. The alphanumeric value you need is labeled as the Account Key. 

After you start getting machines registered you will see them automatically show up under My Contribution and can click on Device Statistics.

The ansible device name seen in this screenshot is the control node of different student topologies. I have had this Ansible Role only live in our production workloads for an hour or so and I am already seeing over a hundred instances crunching through World Community Grid data.

 

Takeaways

As an IT infrastructure engineer it can be difficult to come up with ideas on how we can help our community. I am trying to brainstorm ideas how I can use my existing skills to help contribute to making our world a better place. World Community Grid seemed like an easy place for us to contribute.

By installing the agent into our workshop workloads we can help with their cause. Ansible Automation is simply a tool here to help install the software in masse. If you can come with other ideas please let us know! https://www.ansible.com/contact-us or attend our Here to Help webinar series: https://www.ansible.com/here-to-help-webinar-series