Random Gleanings

Using R to Work with Fitbit Intraday Data

I asked for a Fitbit Charge for Christmas. I always enjoy new data sets, and I’m interested in the so called “quantified self” movement, i.e. gathering data about oneself in order to monitor all kinds of things such as blood pressure and heart rate. The Fitbit Charge sounded ideal as it would provide new data for me to play with, and the data would be about my health. For example, I am particularly interested in the new intraday heartrate data available at one minute granularity that Fitbit recently made available via its API.

In the spirit of Christmas giving, alongside the qualifier to my family that I would buy it myself if I didn’t receive it as a Christmas present, I found a new Fitbit Charge under the tree. I immediately wanted to start working with the intraday heart rate data. I thought I would setup R code to interact with the Fitbit API. Being lazy, my initial thought was to Google a way to connect R to the Fitbit API and grab the intraday heart rate data. However, because Fitbit only recently modified the API to provide the intraday data and because the Fitbit API authorization doesn’t seem to conform precisely to the typical OAuth2.0 usage within R, I found I had to pave my own way. This turned out to be quite an adventure, involving working with a number of cool concepts that were new to me within the R environment. For example:

OAuth2.0 security from within R
using HTTP “GET”, “POST” and headers within R
working with JSON files returned by the Fitbit API
encoding, e.g. working with data encoded as base64 and raw bytes

I thought I’d share my struggle here for your enjoyment, to potentially help out anyone else with a Fitbit trying to use R to work with the relatively new Fitbit intraday data and most importantly to see if folks can point us towards a tighter and less hack-y approach.

Background

This post is specific to the Fitbit intraday heart rate data, so for this to be relevant the reader must own a Fitbit Charge or another of their models that monitor heart rate. However, the basic concepts extend to other data as well which are provided by other Fitbit models, e.g. basic data such as number of steps taken per day.

In a nutshell, the Fitbit data stream starts with the Fitbit device on your wrist that captures data about you. The device caches the data and sends to a Fitbit app (in my case on my iPhone) whenever ones synchs the app and the device, The application on the mobile device sends the data to the Fitbit servers. Finally, a Fitbit user can use the official Fitbit website or the Fitbit API to access the data from the Fibit servers.

If one wants to work with the data in a third party environment such as R, one needs to use the Fitbit API to register a FitBit “application”. See diections to do this at https://dev.fitbit.com/docs.

Once you have registered a FitBit “application”“, you will have all the security parameters you need to access the Fitbit data via OAuth2.0 in R, which we’ll show in detail below.

R Code

I used knitr for this write-up to provide the R code and sample output.

Note: I created and then deleted a Fitbit app so I could show all the private Fitbit application parameters that one would not typically show on a blog. I hope this makes for a better exposition.

R Libraries

library(httr) ## for interacting with the Fitbit API over the web
library(base64enc) ## authorization gymnastics requires some encoding
library(jsonlite) ## Fitbit API returns JSON file
library(ggplot2) ## have fun with the data when we finally get it

Fitbit Application Parameters

The Fitit API provides these parameters when one registers an application at https://dev.fitbit.com/apps/new. We’ll use these in the OAuth2.0 gymnastics below

fitbit_authorization_url = "https://www.fitbit.com/oauth2/authorize"
fitbit_access_url = "https://api.fitbit.com/oauth2/token"
fitbit_app <- "testBits"
fitbit_app_redirect_url = "http%3A%2F%2Flocalhost:1410"
fitbit_app_secret = "1117267a9dc748f00631915be59ff409"
fitbit_app_clientID <- "227HVW"

OAuth2.0 Step 1: Request an API Authorization Code

For background on OAuth2.0 and the Fitbit API see https://dev.fitbit.com/docs/oauth2

As far as I can gather from searching around the web, my use here of the OAuth2.0 tools in the “httr” package is a bit unconventional. For example, I could not get oauth_app() nor oauth2.0_token() to work for Fitbit as I have seen it used with, for example, the Github or Google APIs. As such, my hack is to provide the authorization URL the Fitbit API requires to httr::oauth_listener() directly.

In the R code I break up the URL into managable chunks. The components of the URL are as required by the Fitbit API. See https://dev.fitbit.com/docs

authorizeUrl_Prefix <- "https://www.fitbit.com/oauth2/authorize?response_type=code"
authorizeUrl_ClientID <- paste0("&client_id=", fitbit_app_clientID)
authorizeUrl_Redirect <- paste0("&redirect_uri=",fitbit_app_redirect_url)
authorizeUrl_Scope <- "&scope=activity%20heartrate%20location%20sleep"
authorizeUrl <- paste0(authorizeUrl_Prefix,authorizeUrl_ClientID,
                       authorizeUrl_Redirect,authorizeUrl_Scope)
authorizeUrl

## [1] "https://www.fitbit.com/oauth2/authorize?response_type=code&client_id=227HVW&redirect_uri=http%3A%2F%2Flocalhost:1410&scope=activity%20heartrate%20location%20sleep"

Note: at this point one way to test whether the URL you have built will work is to simply paste the URL above into your browser. The Fitbit API should return a page asking you to allow access, which will in turn send you to the redirect URL with a authorization code.

Back to R. In the code snippet below we simply tell httr::oauth_listener() we are asking for an authorization code to return the relevant code in the variable “req”. Note a webpage will pop-up with Fitbit asking you to grant the access R has requested. Simply click OK and the control will return to R. In our case the result was returned to our variable “req” and the Fitbit authorization code will reside in “req$code”.

req <- oauth_listener(authorizeUrl, is_interactive = interactive())
fitbitAuthorizationCode <- req$code

Note that, of course, this authorization code will be different each time one requests authorization. Just showing here as an example.

fitbitAuthorizationCode

## [1] "f22cef769f2f26ae54eb996e13f46b3fa21cf2cc"

OAuth2.0 Step 2: Use Authorization Code to Request Access Token

Now that we have received the OAuth2.0 authorization code, the OAuth2.0 gymnastics requires that we exchange the authorization code plus some of our FitBit app specific info for an access token. We need to use HTTP to POST our request for an access token and receive the result in an R variable. First we build the URL we will POST. Once again, the components of the URL are as required by the Fitbit API. See https://dev.fitbit.com/docs

authorizePost_Prefix <- fitbit_access_url
authorizePost_ClientID <- paste0("client_id=", fitbit_app_clientID)
authorizePost_GrantType <- "&grant_type=authorization_code"
authorizePost_Redirect <- paste0("&redirect_uri=",fitbit_app_redirect_url)
authorizePost_GrantCode <- paste0("&code=", fitbitAuthorizationCode)
authorizePost_Body <- paste0(authorizePost_ClientID,authorizePost_GrantType,
                             authorizePost_Redirect,authorizePost_GrantCode)

Within this POST, the Fitbit API requires that We provide an HTTP header that contains some of our fitbit application parameters (the fitbit application ID and the Fitbit API “secret”). A twist here is that we are required to base 64 encode this info in the header.

base64encodedData <- base64encode(charToRaw(paste0(fitbit_app_clientID, ":", 
                                                   fitbit_app_secret)))
myAuthorization <- paste0("Basic ", base64encodedData)
myAuthorization

## [1] "Basic MjI3SFZXOjExMTcyNjdhOWRjNzQ4ZjAwNjMxOTE1YmU1OWZmNDA5"

Now that we have an URL and the required authorization header, we can go ahead and POST. If all goes well we will see a Status: 200 in the abbreviated JSON response to the POST

req <- POST(authorizePost_Prefix, add_headers(Authorization = myAuthorization,
                   "Content-Type" = "application/x-www-form-urlencoded"),
            body = authorizePost_Body)
req

## Response [https://api.fitbit.com/oauth2/token]
##   Date: 2016-01-05 05:15
##   Status: 200
##   Content-Type: application/json;charset=UTF-8
##   Size: 445 B

Inside of the “content” field of the JSON file we have a number of pieces of information, including the access token we are after as well as token refresh code and other data. This is all provided in one long character variable (once converted from raw format). Here’s an example of what the character looks like. Again, this character will be different every time and this is just an example.

rawToChar(req$content)

## [1] "{\"access_token\":\"eyJhbGciOiJIUzI1NiJ9.eyJleHAiOjE0NTE5NzQ1MDQsInNjb3BlcyI6InJociBybG9jIHJzbGUgcmFjdCIsInN1YiI6IjNaVDJLTCIsImF1ZCI6IjIyN0hWVyIsImlzcyI6IkZpdGJpdCIsInR5cCI6ImFjY2Vzc190b2tlbiIsImlhdCI6MTQ1MTk3MDkwNH0.TxSwbqHjeSkIK8f87Izf8gdEPY902c7nAGyCGRCusko\",\"expires_in\":3600,\"refresh_token\":\"91b00cc8c02b8848d7bcd3512dae0848b9af7539aa5e98488b95784400134aec\",\"scope\":\"activity location heartrate sleep\",\"token_type\":\"Bearer\",\"user_id\":\"3ZT2KL\"}"

How can we get the access token out of this character variable? One qick and dirty approach is to simply hard wire the start and end position of the authorization token and use substr() to extract the part we need. Hallelujah, we have our access token

startTokenSubstring <- 18
endTokenSubstring <- 257
fitbitTokenString <- substr(rawToChar(req$content), start = startTokenSubstring,
                            stop = endTokenSubstring)
fitbitToken <- paste0("Bearer ", fitbitTokenString)
fitbitToken

## [1] "Bearer eyJhbGciOiJIUzI1NiJ9.eyJleHAiOjE0NTE5NzQ1MDQsInNjb3BlcyI6InJociBybG9jIHJzbGUgcmFjdCIsInN1YiI6IjNaVDJLTCIsImF1ZCI6IjIyN0hWVyIsImlzcyI6IkZpdGJpdCIsInR5cCI6ImFjY2Vzc190b2tlbiIsImlhdCI6MTQ1MTk3MDkwNH0.TxSwbqHjeSkIK8f87Izf8gdEPY902c7nAGyCGRCusko"

Use Access Token and HTTP to GET our Intraday Heartrate Data

Now that we finally have our access token, we can send a request for our heart rate data using an HTTP GET request, supplying the access token data as a header in the GET statement.

Break the URL up into reasonable chunks.

heartRateUrlPrefix <- "https://api.fitbit.com/1/user/-/activities/heart/date/"
heartRateUrlSuffix <- "/1d/1min/time/00:00/23:59.json"
heartRateUrlDate <- "2015-12-26"
hearRateURL <- paste0(heartRateUrlPrefix,heartRateUrlDate,heartRateUrlSuffix)
hearRateURL

## [1] "https://api.fitbit.com/1/user/-/activities/heart/date/2015-12-26/1d/1min/time/00:00/23:59.json"

If all goes well we will see a Status: 200 in the abbreviated JSON response to the GET

getHeartRateOneDay <- GET(hearRateURL, 
                             add_headers(Authorization = fitbitToken)) 
getHeartRateOneDay

## Response [https://api.fitbit.com/1/user/-/activities/heart/date/2015-12-26/1d/1min/time/00:00/23:59.json]
##   Date: 2016-01-05 05:15
##   Status: 200
##   Content-Type: application/json;charset=UTF-8
##   Size: 43.2 kB

The Fitbit API returns the data in JSON format. It includes a number of fields as we can see below. We need to extract time and heart rate from list object in turn derived from JSON object

listData <- fromJSON(rawToChar(getHeartRateOneDay$content))
str(listData)

## List of 2
##  $ activities-heart         :'data.frame':   1 obs. of  4 variables:
##   ..$ customHeartRateZones:List of 1
##   .. ..$ : list()
##   ..$ dateTime            : chr "2015-12-26"
##   ..$ heartRateZones      :List of 1
##   .. ..$ :'data.frame':  4 obs. of  5 variables:
##   .. .. ..$ caloriesOut: num [1:4] 1496 842 291 629
##   .. .. ..$ max        : int [1:4] 85 119 145 220
##   .. .. ..$ min        : int [1:4] 30 85 119 145
##   .. .. ..$ minutes    : int [1:4] 1108 190 30 46
##   .. .. ..$ name       : chr [1:4] "Out of Range" "Fat Burn" "Cardio" "Peak"
##   ..$ value               : chr "71.15"
##  $ activities-heart-intraday:List of 3
##   ..$ dataset        :'data.frame':  1374 obs. of  2 variables:
##   .. ..$ time : chr [1:1374] "00:00:00" "00:01:00" "00:02:00" "00:03:00" ...
##   .. ..$ value: int [1:1374] 64 63 63 63 62 62 62 62 63 64 ...
##   ..$ datasetInterval: int 1
##   ..$ datasetType    : chr "minute"

Upon inspection one can see that the list element we want to focus on to get the minute-by-minute heart rate data is activities-heart-intraday$dataset

myTime <- listData$`activities-heart-intraday`$dataset$time
myValues <- listData$`activities-heart-intraday`$dataset$value
myDateString <- listData$`activities-heart`$dateTime

plotX <- as.POSIXlt(myTime, format = "%H:%M:%S")  
plotY <- myValues
myPlot <- ggplot(data = data.frame(x = plotX, y = plotY), aes(x = x, y = y))
myPlot <- myPlot + geom_line()
myPlot <- myPlot + xlab("Time of Day (minutes)") + ylab("Heart Rate (bpm)") + 
  ggtitle(paste0("Minute Level Heart Rate from Fitbit for: ", myDateString))
myPlot

Conclusion

In a bit of a hack, we’re up and running with the Fitbit intradday data in R.

A lot more to do, e.g. writing code to plot multiple days, to plot an expanding window, applying some cool analysis to the data, combining with other “quantified self” style data, etc.

A lot more to learn about OAuth2.0, e.g. how to refresh the token and how to use the httr library with the Fitbit API in a more conventional manner.

Here’s to a Happy & Healthy 2016!