Dev Update (2019-11-09)

I’m finally doing my Friday dev updates. These posts are short and sweet – they talk about work that went into any of my open source projects for the past ~week and/or what work is being done with them.

Let’s dive in. Continue reading “Dev Update (2019-11-09)”

CommunityPatch.com (beta)

In previous posts, I talked about two projects I had been working on for the Jamf community to better help admins get started using the new “External Patch Sources” feature in Jamf Pro 10.2+. While working on Patch Server and the companion Patch-Starter-Script, I also wrote a quick proof of concept for a serverless version that would run in an AWS account.

The Stupid Simple Patch Server uses API Gateway and Lambda functions to serve patch definitions you stored in an S3 bucket. I even included the same API endpoints from the Patch Server so workflows between the two could be shared. I even took it a step further and added a subscription API so it would sync with a remote patch definition via URL.

That side project (of a side project) made me think about how I could take the basic design and build upon it into something that could be used by multiple admins. At first, I wrote a lot of code to transform the Stupid Simple Patch Server into a multi-tenant application. At a point, I considered the limitations of what could be done in a manner that could be considered secure and scrapped much of it.

But not everything. The work I had done was retooled into a new concept: a single, public, community managed patch source for Jamf Pro. A service where anyone could contribute patch definitions, and be able to manage and update them. Five minutes after having this idea I bought the communitypatch.com domain and setup a beta instance of my work-in-progress:

https://beta.communitypatch.com

CommunityPatchBeta.png

New API

The big green “Read the docs” button on the main page will take you to… the documentation! There you will find those APIs in much greater detail.

The community managed patch source mirrors a number of features from my Patch Server project. The /jamf endpoints are here to integrate with Jamf Pro and the service can be used as an external patch source.

The /api endpoints are slightly different from the Patch Server, but allow for creating definitions by providing the full JSON or a URL to an external source (creating a synced definition) and updating the versions afterwards.

From the docs, here’s the example for creating a new patch definition using the Patch-Starter-Script:

curl https://beta.communitypatch.com/api/v1/title \
   -X POST \
   -d "{\"author_name\": \"<NAME>\", \"author_email\": \"<EMAIL>\", \"definition\": $(python patchstarter.py /Applications/<APP> -p "<PUBLISHER>")}" \
   -H 'Content-Type: application/json'

Here, there are required author_name and author_email keys you need to provide when creating a definition. The author_name you choose will be injected into the ID and name keys of the definition you’re providing.

For example, if I provide “Bryson” for my name, and I’m creating the “Xcode.app” definition, it’s ID will become “Xcode_Bryson” and the display name “Xcode (Bryson)”. These changes make it possible to differentiate titles when browsing in Jamf Pro, and for members of the community to better identify who is managing what (as well as sharing with each other).

After you create a patch definition, you will be emailed an API token to the address you provided in author_email. This token is specifically for managing that title, and is the only way to update the title after. Your email address is not saved with CommunityPatch. A hash of it is stored with the title so you can reset the token should you lose it or need the previous one invalidated (this feature is not implemented yet).

Updating works similarly to Patch Server (but without the items key):

curl http://beta.communitypatch.com/api/v1/title/<ID>/version \
   -X POST \
   -d "$(python patchstarter.py /Applications/<APP> --patch-only)" \
   -H 'Content-Type: application/json' \
   -H 'Authorization: Bearer <TOKEN>'

 

Try It Out

I had a number of admins on Slack giving me feedback and testing the API for a few weeks. While I have work left to do to ensure the production version of CommunityPatch is performant, and still some more features to finish writing, I am at a stage where I would like those interesting in contributing to and using CommunityPatch to join in, and try the documented features (in your test environments).

You can jump right in by joining the #communitypatch channel on the MacAdmins Slack, hitting the CommunityPatch documentation, play around with the API, test definitions you create in your Jamf Pro test environments, and discuss what you find.

CommunityPatch is being written out in the open. You can go to GitHub and see the code for yourself. You can even contribute at a code/docs level if you like! For the immediate, having admins test it out and report back will provide me a lot of value as I work towards completing the application and deploying it to production.

Links

Patch Starter Script

Jamf Pro 10.2 is not far. I recently released my Patch Server project for admins in the beta (and after the release) to host their own custom patch definitions from. The server hosts the definitions and provides an API for maintaining them afterwards with automation. A missing piece is a tool to create your initial definitions to upload.

This morning I posted a script that aims to address that gap:

https://github.com/brysontyrrell/Patch-Starter-Script

This command line utility will take an existing application on your Mac and generate a basic (we’ll call it default) patch definition. This is primarily done using the Info.plist file within the app bundle.

Creating a Patch Definition

Using GitHub Desktop.app as an example, the script can output the JSON to stdout:

$ python patchstarter.py /Applications/GitHub\ Desktop.app -p "Github"

Or it can write a JSON file to a directory of your choice (in this example the current working directory):

$ python patchstarter.py /Applications/GitHub\ Desktop.app -p "Github" -o .

The -p or –publisher argument allows you to give the name of the application’s publisher. This was included as this information is not (normally) found in Info,plist.

Below, I’ve shown the GitHub Desktop.app example’s output (the keys print out of order in Python 2, but keep in mind that key order doesn’t matter). Next to each line I’ve included where the value maps from:

{
    "id": "GitHubDesktop",                          <-- CFBundleName without spaces
    "name": "GitHub Desktop",                       <-- CFBundleName
    "appName": "GitHub Desktop.app",                <-- Application filename
    "bundleId": "com.github.GitHub",                <-- CFBundleIdentifier
    "publisher": "GitHub",                          <-- Optional command line argument (see above)
    "currentVersion": "Hasty Things Done Hastily",  <-- CFBundleShortVersionString
    "lastModified": "2018-02-12T18:33:02Z",         <-- UTC Timestamp of when the script ran
    "requirements": [
        {
            "name": "Application Bundle ID", 
            "operator": "is", 
            "value": "com.github.GitHub",  <-- CFBundleIdentifier
            "type": "recon", 
            "and": true
        }
    ], 
    "patches": [
        {
            "version": "Hasty Things Done Hastily",  <-- CFBundleShortVersionString
            "releaseDate": "2017-05-22T20:24:33Z",   <-- Application last modified timestamp
            "standalone": true, 
            "minimumOperatingSystem": "10.9",        <-- LSMinimumSystemVersion
            "reboot": false, 
            "killApps": [
                {
                    "appName": "GitHub Desktop.app",  <-- Application filename
                    "bundleId": "com.github.GitHub"  <-- CFBundleIdentifier
                }
            ], 
            "components": [
                {
                    "version": "Hasty Things Done Hastily",  <-- CFBundleShortVersionString
                    "name": "GitHub Desktop",                <-- CFBundleName
                    "criteria": [
                        {
                            "name": "Application Bundle ID", 
                            "operator": "is", 
                            "value": "com.github.GitHub",  <-- CFBundleIdentifier
                            "type": "recon", 
                            "and": true
                        }, 
                        {
                            "name": "Application Version", 
                            "operator": "is", 
                            "value": "Hasty Things Done Hastily\",  <-- CFBundleShortVersionString
                            "type": "recon" 
                        }
                    ]
                }
            ], 
            "capabilities": [
                {
                    "name": "Operating System Version", 
                    "operator": "greater than or equal", 
                    "value": "10.9",  <-- LSMinimumSystemVersion
                    "type": "recon"
                }
            ],
            "dependencies": []
        }
    ],
    "extensionAttributes": []
}

It is important to understand how the above values map in the event that you create a definition, but it has used incorrect values because the developer assigned them differently than what is considered standard (especially true for version strings).

In the event the CFBundleShortVersionString or LSMinimumSystemVersion keys are missing from Info.plist, the script will prompt you for an alternative.

Add Extension Attributes

You can also include extension attributes that need to be a part of your definition. For example, if you have the following bash script as an extension attribute for GitHub Desktop.app saved as github-ea.sh:

#!/bin/bash

outputVersion="Not Installed"

if [ -d /Applications/GitHub\ Desktop.app ]; then
    outputVersion=$(defaults read /Applications/GitHub\ Desktop.app/Contents/Info.plist CFBundleShortVersionString)
fi

echo "<result>$outputVersion</result>"

You can pass it to the -e or –extension-attribute argument:

$ python patchstarter.py /Applications/GitHub\ Desktop.app -p "Github" -e github-ea.sh

The extension attribute will be appended to the extensionAttributes key in the definition (the key here is the appName in lowercase with spaces replacing dashes):

{
    "...",
    "extensionAttributes": [
        {
            "key": "github-desktop",
            "value": "IyEvYmluL2Jhc2gKCm91dHB1dFZlcnNpb249Ik5vdCBJbnN0YWxsZWQiCgppZiBbIC1kIC9BcHBsaWNhdGlvbnMvR2l0SHViXCBEZXNrdG9wLmFwcCBdOyB0aGVuCiAgICBvdXRwdXRWZXJzaW9uPSQoZGVmYXVsdHMgcmVhZCAvQXBwbGljYXRpb25zL0dpdEh1YlwgRGVza3RvcC5hcHAvQ29udGVudHMvSW5mby5wbGlzdCBDRkJ1bmRsZVNob3J0VmVyc2lvblN0cmluZykKZmkKCmVjaG8gIjxyZXN1bHQ+JG91dHB1dFZlcnNpb248L3Jlc3VsdD4iCg==",
            "displayName": "GitHub Desktop"
        }
    ]
}

This will not update any of the capabilities or components/criteria in the generated definition! It will be up to you to make the edits to reference the extension attribute.

Create Patch Data Only

This script is only meant to create a starter definition for an application. It will only have the one version so there will be no historical data for reporting. However, if you upload it to your own running Patch Server you can maintain it by updating the version through the API.

The –patch-only argument will take an application and only generate that portion of the definition:

python patchstarter.py /Applications/GitHub\ Desktop.app -p "GitHub" --patch-only

The JSON output can be sent to the Patch Server’s API to update the stored definition. An example curl command is in the GitHub readme:

$ curl -X POST http://localhost:5000/api/v1/title/GitHubDesktop/version -d "{\"items\": [$(python patchstarter.py /Applications/GitHub\ Desktop.app -p "GitHub" --patch-only)]}" -H 'Content-Type: application/json'

Try It Out

This script should ease the process of getting started with your own custom patch definitions. If you have access to multiple versions of an application, you can use the –patch-only argument to generate the data for each and place them into your starter definition.

Note that versions must be in descending order! The latest version must always be at the top of your patches array and the older version at the bottom.

If you run into problems or have feature requests, open an issue on the GitHub repository!

Patch Server for Jamf Pro

(TL;DR, gimme the link: https://github.com/brysontyrrell/PatchServer)

After several months of not getting around to it, my PatchServer project on GitHub is finally nearing a true 1.0 state.

I am openly asking for those who have been following this project, and those who are interested in this project, to download, use, and provide feedback on what should be finished before the release of Jamf Pro 10.2.

Please create issues on GitHub for bugs and feature requests that you would want to make the cut for 1.0.

Some time late last year (and I say some time because it’s all becoming a blur), I was brought into a meeting where I was shown our (Jamf’s) progress on providing a framework for customers to be able to create their own patch definitions. This framework would allow customers to setup their own patch servers and add them to their JSS.

A day or so later, I wrote the first rough version of my own implementation.

Backing up a sec:

What’s a patch definition?

In Jamf Pro v10 we introduced a feature called Patch Management. With this, you could subscribe to a number of software titles that Jamf curates and maintains. Once subscribed, your JSS will, on a schedule, read in the patch definitions of those software titles to stay updated.

For more about Patch Management, see the Jamf Pro Admin Guide (10.1):

These patch definitions (which are JSON data) contain historical information about a software title’s version history and requirements for determining if the software is installed on a managed computer. This allows admins to use the Patch Management feature to create reports and update policies to automatically patch those software titles on computers.

Of course, when these features came out there was one resounding question from nearly everyone:

“Why can’t we make our own patch definitions?”

External Patch Sources

The framework I mentioned above is the answer to this. In Jamf Pro 10.2+ you will have the option of adding External Patch Sources to your JSS. Then, in addition to the official Jamf software titles, you will be able to subscribe to your own and use the same reporting and policy features.

 

The external patch source must be a server your JSS is able to reach via HTTP/HTTPS. This patch server must expose the following endpoints:

  • /software
    This returns a JSON array of all the software titles that are available on this server. For example:

    [
      {
        "currentVersion": "10.1.1", 
        "id": "JamfAdmin", 
        "lastModified": "2018-02-03T03:34:34Z", 
        "name": "Jamf Admin", 
        "publisher": "Jamf"
      }, 
      {
        "currentVersion": "10.1.1", 
        "id": "JamfImaging", 
        "lastModified": "2018-02-03T03:34:36Z", 
        "name": "Jamf Imaging", 
        "publisher": "Jamf"
      }, 
      {
        "currentVersion": "10.1.1", 
        "id": "JamfRemote", 
        "lastModified": "2018-02-03T03:34:40Z", 
        "name": "Jamf Remote", 
        "publisher": "Jamf"
      }
    ]
  • /software/TitleId,TitleId
    This returns the same JSON as above, but limited to the comma separated list of software titles. For example (passing JamfAdmin,JamfRemote):

    [
      {
        "currentVersion": "10.1.1", 
        "id": "JamfAdmin", 
        "lastModified": "2018-02-03T03:34:34Z", 
        "name": "Jamf Admin", 
        "publisher": "Jamf"
      }, 
      {
        "currentVersion": "10.1.1", 
        "id": "JamfRemote", 
        "lastModified": "2018-02-03T03:34:40Z", 
        "name": "Jamf Remote", 
        "publisher": "Jamf"
      }
    ]
  • /patch/TitleId
    This returns the full patch definition JSON of the software title. Here is an abbreviated example:

    {
      "id": "JamfAdmin",
      "name": "Jamf Admin",
      "publisher": "Jamf", 
      "appName": "Jamf Admin.app", 
      "bundleId": "com.jamfsoftware.JamfAdmin", 
      "currentVersion": "10.1.1", 
      "lastModified": "2018-02-03T03:34:34Z", 
      "extensionAttributes": [
        {"ExtensionAttributeObjects"}
      ],
      "patches": [
        {"PatchObjects"}
      ], 
      "requirements": [
        {"RequirementsObjects"}
      ]
    }

If you had a patch server located at http://patch.my.org, the full URLs would be:

At this time, there is no product that Jamf is providing for customers to install and have a ready to use patch server. The focus has been on opening up the framework that the official patch source uses and allow customers to extend their environments through a little engineering work.

Not all of us are engineers, of course. Thus…

Enter: Patch Server

gui_01_index.png

I wanted to have a working patch server ready for the Jamf community in time for 10.2’s release. My initial patch server implementation (I call it an implementation because it’s one way of providing a patch source) achieved serving the proper JSON data for each of the endpoints described above using a database (SQLite) for the backend.

While my original goals were much grander, including the ability to fully manage a patch definition in a GUI instead of writing out JSON, I had to pare it back in order to get the project into a deliverable state.

In the past week I went through the code and ripped out everything that I felt was not needed, or doable. Then, I went through and added in new features (ported from another project) and streamlined the UI elements that were left.

This patch server features:

  • All required Jamf Pro endpoints to serve as an External Patch Source
  • An API for programmatic management of patch definitions and versions.
    • Create/delete patch definitions.
    • Add versions to existing patch definitions.
    • Create backup archives of all patch definitions.
  • UI for management of patch definitions.
  • Validation of uploaded patch definitions.
    gui_05_validation.png
  • Full user documentation at http://patchserver.readthedocs.io/
    patchserver_docs.png

    • UI Overview
    • Setup Instructions
    • API Documentation

Bring the Requests

Until Jamf Pro 10.2 is released, I’m not going to tag the project at a 1.0 version. If you are in Jamf’s beta program and testing 10.2, I invite you to give this a try and let me know what you think. Specifically, I’m asking for you do open up issues on GitHub for:

  • Bugs you find
  • Features you want, such as:
    • Connect to an actual database like MySQL (?)
  • Documentation you want, such as:
    • Instructions for installing on X

Not everything that is reported might get worked on, but the good news is I released the patch server under the MIT license. If you have some Python chops you can fork it and do whatever you want with the codebase to suit your needs!

But, I don’t wanna setup a server…

If you had that reaction to the idea of setting up your own external patch source, ask yourself if you match any of these descriptions:

  1. My JSS can talk to pretty much anything if I want it to,
  2. I want a patch server; I don’t want to host a patch server,
  3. It doesn’t matter where my patches live as long as I can get and manage them,
  4. Can’t this be a cloud thing?

If so… stayed tuned for a future blog post.