GitHub Prometheus Export

** Warning **

This service is currently in public beta. I make no promises regarding its availability, security or reliability. USE AT YOUR OWN RISK and make sure you have read the FAQ section at the bottom.

How to

1. Go to GitHub Developer Settings / Token and obtain a new personal access token. For scraping public repositories, you don't need to select any additional scope.
2. Warning: You should probably NEVER trust me (or anyone) with your GitHub token like this. Use at your own risk! And again, please read the FAQ.
3. Create a static scrape target in your prometheus configuration like this:

   scrape_configs:
   - job_name: CHANGE_ME
     scrape_interval: 2m
     basic_auth: 
       username: YOUR-GITHUB-USERNAME
       password: YOUR-ACCESS-TOKEN
     scheme: https
     metrics_path: /YOUR-GITHUB-USERNAME/YOUR-REPOSITORY
     static_configs:
       - targets: ['gh.skuzzle.de']

   Note: You probably don't want to put your token in plain text into the config. Prometheus also allows to read it from a mounted secret file.

   Note: You can scrape multiple repositories of the same owner with a single job. Just list the repository names separated with a comma like this: metrics_path: /YOUR-GITHUB-USERNAME/YOUR-REPOSITORY1,REPOSITORY2

FAQ

• Why do I need to use a GitHub access token?

  GitHub enforces a rate limit to its API. If we'd query the API anonymously, we would be restricted to at most 60 requests per hour from our IP addresses. By using access tokens, every API call will be counted against YOUR personal hourly rate limit (which is 5000 requests per hour).

• How much of my personal API rate limit will you consume?

  Independent of the scrape interval you specified in the prometheus configuration, we will actually only scrape the GitHub repository every 30 minutes to not stress your user dependent GitHub API rate limit. A single scrape takes like 2 API calls. In the meanwhile, we return cached metrics from the most recent repository scrape.

• I have security concerns trusting you with my access token

  Its ok, you should have. In a perfect world, we'd use a GitHub OAuth client or GitHub App which you'd had to authorize for specific repositories. But this hasn't been implemented yet as this is, at the time of writing, just an experimental side project. If demand rises, I might ship this as a feature.

  Until now, your only chance is to believe that the following is true:
  

  Your access token is neither stored nor logged anywhere in this application. There is no technical possibility it can be determined by anyone from the requests you make to gh.skuzzle.de.

• How often can I call your service?

  There is currently no rate limit employed for calls to this service that complete successfully. However, if calls from your IP do cause more than a few exceptions (for example if you provide an invalid access token or unknown scrape target) within an unspecified amount of time, your IP will be blocked for an unspecified amount of time.

• Can I monitor private repositories?

  Technically: yes. But concerning security, you probably don't want to pass around GitHub tokens with access to your private repositories.

• I don't want to use the public service. Can I host this on-premise?

  Yes, you can run this exact application yourself using Docker. You can find a detailed explanation on GitHub.

• How can I visualize the scraped metrics in grafana?

  I've created a very basic dashboard for this. You can download its json model right here.

• Which metrics are reported?

  Table of metrics that are being reported to prometheus during each scrape

  Metric	Description
  github_additions_total	Sum of additions over the last 52 weeks
  github_deletions_total	Negative sum of deletions over the last 52 weeks
  github_stargazers_total	Number of stargazers per repository
  github_forks_total	Number of forks per repository
  github_open_issues_total	Number of open issues per repository
  github_subscribers_total	Number of subscribers per repository
  github_watchers_total	Number of watchers per repository
  github_size_total	Size in KB of the repository
  github_scrape_duration_sum	Scrape duration in ms per repository