Exploring Everyday Things with R and Ruby.

Jul 31, 2012 4:45:41 PM | Exploring Everyday Things with R and Ruby.

Exploring Everyday Things with R and Ruby.

This is an invited guest post from Sau Sheong Chang. After meeting Sau we fell in love with his creative love of Ruby and R. We have an exclusive preview of his new book, Exploring Everyday Things with R and Ruby: Learning About Everyday Things [Paperback].

Getting your heart rate using R and Ruby

The heart rate, or the rate at which your heart beats, is one of the measurements you’ve probably heard most about in relation to exercise. It’s also often a good indication of your health, because a heart rate that is too high or low couldindicate an underlying health issue. The heart rate is usually measured in beats per minute (bpm) and varies from 40 to 220 bpm. An average healthy person at rest has a heart rate of 60–90 bpm, while conditioned atheletes have a resting heart rate of 40– 60 bpm.

A popular and fast way to effectively get the heart rate is pulse oximetry. A pulse oximeter is a device placed on a thin part of a person’s body, often a fingertip or earlobe. Light of different wavelengths (usually red and infrared) is then passed through that part of the body to a photodetector. The oximeter works by measuring the amounts of red and infrared light absorbed by the hemoglobin and oxyhemoglobin in the blood to determine how oxygenated the blood is. Because this absorption happens in pulses as the heart pumps oxygenated blood throughout the body, the heart rate can also be determined.

We are not going to build an oximeter, but in this post we’ll use the same concepts used in oximetry to determine the heart rate. We will record a video as we pass light through our finger for a short duration of time. With each beat of the heart, more or less blood flows through our body, including our finger. The blood flowing through our finger will block different amounts of the light accordingly. If we calculate the light intensity of each frame of the video we captured, we can chart the amount of blood flowing through our finger at different points in time, therefore getting the heart rate.

Homemade Pulse Oximeter

Creating a homemade oximeter is really simple. You can use any of the following techniques, or even try your own methods. It doesn’t really matter, as long as you can capture the video. Record for about 30 seconds. (Recording for a longer time can be more accurate, but not significantly so.)

Finger on a webcam

Place your finger directly on your computer’s webcam (I used the iSight on my Mac). Shine a small light (penlight or table lamp; it doesn’t matter much) through your finger. Then use any video recording software to record what’s on the web­ cam (I used QuickTime video recording).

Finger on the phone camera

Place your finger directly on your phone camera. Turn on the flash or use a small light and shine it through your finger. Then use your phone’s video recording software to record what’s on the phone camera.

Finger on a digital video camera

This is slightly harder because the camera lens is normally larger than your finger. The parts that aren’t covered don’t really matter, but you need to position your finger so that the image captured is consistent throughout your recording. A trick is to use a lamp as the background, so you can have the light shining through your finger and maintain a consistent background at the same time.

In the following example, I used the phone camera method with my iPhone. That’s the easiest for me, because the flash on the phone is very effective. If you did things right, you’ll end up with a video filled with a red blotch that’s your finger.

Extracting Data from Video

Assuming that you have a nice video file now (it doesn’t really matter what format it is in; you’ll see why soon), let’s dig in a bit deeper to see how we can extract infor­ mation from it. For the sake of convenience, I’ll assume the file is called heart beat.mov. Next we’ll be using FFmpeg, a popular free video library and utility, to convert the video into a series of individual image files.

Let’s take a look at some Ruby code.

  require 'csv'
require 'rmagick'
require 'active_support/all'
require 'rvideo'
vid = RVideo::Inspector.new(:file => "heartbeat.mov")
width, height = vid.width, vid.height
fps = vid.fps.to_i
duration = vid.duration/1000
if system("/opt/local/bin/ffmpeg -i heartbeat.mov -f image2 'frames/frame%03d.png'")
CSV.open("data.csv","w") do |file|
file << %w(frame intensity)
(fps*duration).times do |n|
img = Magick::ImageList.new("frames/frame#{sprintf("%03d",n+1)}.png")
ch = img.channel(Magick::RedChannel) i= 0
ch.each_pixel {|pix| i += pix.intensity} file << [n+1, i/(height*width)]
end end
end

It doesn’t look complicated, does it? The most complex part you’ll probably have to tackle is installing the necessary Ruby libraries. In the case of both RMagick and RVideo, described next, you need native developer tools support in order to compile the native components of the gem for your platform.

We start off the code by inspecting the video and getting some attributes from it. These will be useful later on in the code. Specifically, we will need the number of frames per second, the duration of the video, and the height and width of the video. You can obtain these through RVideo, but if you didn’t succeed in getting it installed, you can still find the information by simply opening up the video with any player and viewing its properties.
Next, we use the system method to issue a command to the underlying shell, and return either true or false depending on whether it succeeds or not:

system("/opt/local/bin/ffmpeg -i heartbeat.mov -f image2 'frames/frame%03d.png'")

This runs ffmpeg, taking in the input file heartbeat.mov and converting it frame by frame into a set of images ordered by number. This is the reason why the video format is unimportant. As long as FFmpeg has the correct library to support the codecs, it will convert the video file to a series of PNG image files, numbered sequentially.

In this example, we specify that there are three digits to this series of numbers. How do we know this? In my case, I have a 30-second video with a frame rate of 30 frames per second, so the number of still frames that will be created by FFmpeg is 30×30, or 900 frames. Slightly more frames could be created—some video players round off the duration—but the total would not be more than 999 frames. If the command runs successfully, we will get a set of frames in the frames folder, each named framennn.png, where nnn runs from 001 to 900 or so.

Next, we create a CSV file to store the data and enter the column names, which are the frame number and the average frame intensity:

file &lt;&lt; %w(frame intensity)

Then, for every frame image, we create the RMagick Image object that represents that frame and extract the red channel (the file uses the RGB colorspace):

  ch = img.channel(Magick::RedChannel)

We iterate through each pixel in the red channel and add up their intensities, then divide the sum of pixel intensities by the total number of pixels:

    i= 0
ch.each_pixel {|pix| i += pix.intensity}
file << [n+1, i/(height*width)]

This is the value we consider to be the average frame intensity. Finally, we store the frame number and intensity in the CSV file.

Once we have done this, we will end up with a data file with two columns. The first is the frame number, and the second is the corresponding frame’s average intensity.

Generating the Heartbeat Waveform and Calculating the Heart Rate

Generating the heartbeat waveform is trivial, so we’ll combine both creating the waveform and calculating the heart rate into a single R script.

 library(PROcess)
library(ggplot2)
data <- read.csv(file='data.csv', header=T)
png("heartbeat.png")
qplot(data=data, frame, intensity, geom="line")
dev.off()
peaks <- peaks(data$intensity,span=10)
peak_times <- which(peaks==T, arr.in=T)
intervals <- c()
i <- 1
while (i < length(peak_times)) {
intervals <- append(intervals, peak_times[i+1] - peak_times[i])
i <- i + 1
}
average <- round(mean(intervals))
print(paste("Average interval between peak intensities is", average))
heartbeat_rate <- round(60 * (30/average))
print(paste("Heartbeat rate is",heartbeat_rate))

All it takes to generate the waveform is a single line that calls qplot with the frame and the intensity and uses the line geom.

As you can see from the chart, the light intensity changes over time. Each pulse corresponds with a heartbeat. To find the heart rate, we need to find the number of frames between two peaks of the wave. We know that there are 30 frames in one second. Once we know the number of frames between the two peaks, we’ll know how much time it takes to go from peak to peak, and therefore can calculate the number of beats per minute.

To calculate the distance from peak to peak, we need to first determine where the peaks are in the chart. For this, we will be using an R package that was originally designed to process protein mass spectrometry data, found in the Bioconductor li­ brary. The Bioconductor library is a free/open source project that provides tools for analyzing genomic data. It’s based primarily on R, and most of the Bioconductor components are R packages. The package we will be using is called PROcess. Once we include the library in our script, we can start using the peaks() function, which, true to its name, determines which values are peaks in data.

The input parameter to the peaks() function is the intensity data and a span value. This span value determines how many of its neighboring values it must exceed before it can be considered a peak. This is useful to filter off noise, though not perfectly.


The returned result is a logical vector that is the same length as the data. This means we have a vector of TRUEs and FALSEs, where the TRUEs indicate a peak:

[1] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[13] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[25] FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE

While this vector is informative, it’s not really the answer we want, so we pass it through the which() function, and it returns a vector of the indices where the element is TRUE:

     [1]  28  50  73  96 119 142 167 190 213 236 259 282 306 330 353 374 397 420 445
[20] 469 494 517 540 563 586 610 632 656 678 701 723 746 769 791 812 836 859 882

As before, we want to find the distance between the two peaks, so we take two con­ secutive elements and subtract the first from the second. This gives us a new vector that contains the differences:

[1] 22 23 23 23 23 25 23 23 23 23 23 24 24 23 21 23 23 25 24 25 23 23 23 23 24
[26] 22 24 22 23 22 23 23 22 21 24 23 23

The final two steps are the same as in the previous section. First, we find the average distance using the mean() function. Then, from that, we know that there are 23 frames between two peaks, meaning each heartbeat takes 23 frames or 23/30 seconds (since each second has 30 frames). From that, we calculate that the heart rate is 78 bpm.

If you enjoyed this blog post we recommend picking up Exploring Everyday Things with R and Ruby: Learning About Everyday Things.

Written By: Frances Banks