Things I Learned Building a Gem

Tech: Ruby, Rubocop, Splunk, Docker, CI, Rspec, TigerConnect

Challenge: a particular component (HIPAA-compliant text messaging) of an application proved valuable in other applications. The code was organically customized and integrated into several applications. Eventually, we had a clear case to pull out the duplicated code (DRY!) from the various repos, standardize, and package as a gem.

What I learned:

  1. send is a Ruby reserved keyword (for dynamically calling method names)
  2. How to pass creds for a private GitLab/GitHub repo into Gemfile, so the gem can be implemented (gitlab-ci-token)
  3. After updating the gem, and using bundle update in an app implementing the gem, sometimes you must force pull from the private repo to get the latest changes.
  4. GitLab CI w/ RSpec, Rubocop!
  5. garbage collection https://blog.codeship.com/visualizing-garbage-collection-ruby-python/
  6. Splunk search is NOT case sensitive!

 

And topics/pitfalls for further research:

  1. Oddly, my Mail settings took in the local environment, but not in the application Docker image utilizing the gem
  2. In the local environment, one dynamic require all in the main module worked nicely for files in ./lib. However, needed multiple require statements (one for each file) when implementing the gem elsewhere.
  3. Ignoring Gemfile.lock in commits was helpful when trying to implement gem in applications.

 

Reference:

Generic Gem Template

Building a Gem Guide

Naming, Versioning, Dependencies, etc.

Bundler – creating gem

GitLab Token

Bug Troubleshooting w/ New Tools

Tech: Ruby, TigerConnect, Splunk, SSH, CSSHX, grep, curl, HTTParty

Challenge: an application is configured to log an event in Splunk after a successful send of a TigerConnect HIPAA-compliant alert message. The entry point is working, however the alerts are not being sent AND the event is not being logged. After successful troubleshooting to determine what’s NOT wrong, I turned to CSSHX and grep to scour the logs. Why CSSHX… our production instances run on 4 servers concurrently. I want to quickly navigate in 1 terminal tab w/ 4 windows and grep the the logs!

Code:

CLI (for 4 instances) => csshx username@dserver.location.extension username@dserver.location.extension username@dserver.location.extension username@dserver.location.extension

 

grep for various strings => grep -A 10 "error" log_file.log

-A # is for number of lines after the string

-B # is for number of lines before

 

I also wanted to check my HTTParty gem code that transmits the event from app to Splunk. I used a curl statement to mimic the HTTParty call.

CLI => curl https://url.extension:####/services/collector -k -H 'content-type: application/json' -H 'authorization: XXXXXX' -d '{"event":{"app": "data"}, "sourcetype": "_json"}'

-k (–insecure) allows for insecure server connections

-H header key, values

 

I was quite proud to implement the use of these tools when troubleshooting. I have used them in different contexts and it was cool to bring everything together to discover the issue. However (and sadly), the issue was much simpler.

Ruby ENV variable are strings. Even if the string is a “boolean”. 

in .env … VAR=false

if ENV['VAR'] == false then <do something> end

=> equates to FALSE… because ENV['VAR'] exists as a string, it is TRUE.

 

Pattern: Iteration within Interation (pt. 2)

Tech: Ruby

Challenge: returning to iterating! Last time, I wanted the sub-iteration to skip records already looked at, as well as skip the index forward. Let’s gooo!

Code:

i = 0

patients.each_with_index do |row, index|

next if i > index # will skip through rows already processed in the while loop

# using while equal makes it a little easier to understand than part 1’s solution

while patients[i][:id] == patients[index][:id]

# i will increase, and as long as it keeps matching our entry index :id; we are within the same patient’s records

# do logic

i += 1

break if !patients[i] # no more records

end

end

 

Takeaway: I previously implemented part 1’s solution, and this solution in production code. I wonder if there is a scenario where part 1 would be optimal comparatively (I think no because rows are redundantly processed).

Ruby => JS: Fill a New Array

Tech: javascript, ruby

Challenge: in a React app, I have to format some data coming from an API… I want to fill out the API data array to a certain length with empty strings. Coming from a Ruby background, I have grown very used to Ruby shortcuts. How do I tackle in JS?

Code:

incoming API data = ['string', 'string']

application needs an array with length 5… like ['string', 'string', "", "", ""]

 

In Ruby, I calculate the difference in lengths and create a new array with empty strings, contained within the .new args:

Array.new(5 - apiArray.length, "")

 

Javascript is just slightly different. Just need to combine new Array with an ES6 prototype!

new Array(5 - apiArray.length).fill("")

 

 

Reference:

https://ruby-doc.org/core-1.9.3/Array.html#method-i-fill

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/fill

 

hsx_hack_3

HSX MarketStreet Hackathon: Victory!

Tech: Ruby on Rails, node JS backend, HSX api

Challenge: Hackathon! The goal was to facilitate the exchange of patient clinical data for the betterment of patients’ health and healthcare management. I am proud to say our team, The DICE Group @ Jefferson, took first place!

https://www.healthshareexchange.org/news/hsx-marketstreet-launches-hackathon-series-successful-innovative-first-hack

hsx_hack_2

 

Our Idea: 

A business-to-business application facilitating practically instant transmittal of pertinent patient data (immunizations, allergies, conditions) that typically takes multiple steps, people and/or points of contact over 1-3 weeks. In addition to time and logistic savings, a target business like a staffing agency typically pays for new tests & immunizations their client has already received but has no records of (like a tetanus booster or measles vaccination).

Target businesses: staffing agencies, college/universities (think incoming freshmen or a student studying abroad), school system/district, day cares, summer camps.

 

Here are some basic screenshots of the web application demonstrating the user interface:

hsx_1

 

 

hsx_2

 

 

hsx_3

 

 

hsx_4

 

And another shot of the winning team!

hsx_hack_1

HealthShare Exchange (HSX): API Connection

Tech: ruby, rails, dotenv, httparty, APIs!

Challenge: connect & query HSX’s API. Querying APIs is relatively new to me, and I wanted to capture the solution to a particular issue I encountered. The authorization POST required the params as a json body (specifically noted in the docs), but the GET required an URL query string (not specified in docs!). Basic, but was confusing for a minute.

Code:

auth with JSON body params

hsx_json_auth

provider query

hsx_get_query

 

Reference:

https://www.healthshareexchange.org/

 

huey

The Power of… Backups!

Code: Heroku, ruby, activerecord

Challenge: database restoration! During my almost-daily updating of www.embiidfeed.com during the NBA season, I accidentally updated ALL the records instead of a specific record. Whoops. I was instantly crushed, angry, and rueful. But much like this blog, present-Luke is very thankful for past-Luke…. say hello to Heroku backups! Luckily, I had taken advantage of Heroku’s daily backup feature and just a few commands would restore the DB to its previous glory. (BTW, I realize consoling into the production database for updates is a bad idea. I have a backlog item to better automate this with a script & endpoint, but personal projects typically take a slower path.)

First, the infraction:

blog_db_backup_1

 

Let’s take a look at what’s available:

blog_db_backup_2

 

And restore!

blog_db_backup_3

 

Takeaway: although not a DB administrator, I will always take a moment to ensure a backup database plan is discussed & implemented for any project! Too dangerous otherwise. And always… Trust the Process.

Reference:

https://devcenter.heroku.com/articles/heroku-postgres-backups

 

 

turtles

Pattern: Iteration within Interation (pt. 1)

Tech: Ruby

Challenge: need to iterate over a set of patient records…within the set, each patient has several records… so when the initial iteration gets to a new patient, the code needs to include a sub-iteration specific to the patient. For example, a patient’s subset of data includes their location & treatment history. We want to capture the patient’s last location, but their first instance of treatment. The data is extracted into a patient-specific data hash.

Code:

patients.each_with_index do |row, index|

i = index
until patients[i]['ID'] != patients[index]['ID']

update_patient_hash(row['ID'], data)

i += 1
break if !patients[i] # no more records

end

end

 

Takeaway: this could benefit from refactoring…

  1. The sub-iteration repeats the processing of records due to reference to the row for data capture, instead of patient[i][data]
  2. Could the initial iteration’s index be reset/moved to the new patient once the sub-iteration is finished?

Ruby Template => Data to JSON

Tech: Ruby, JSON, Docker

Challenge: data for daily reporting is captured in a class instance variable hash. If/when the docker container restarts, I want the data, which is regularly written to a JSON file, to be reloaded into the @hash.

Code:

def initialize

if FileTest.exists?('./data.json')

@instance_var_hash = JSON.parse(File.read('./data.json'))

else

@instance_var_hash = { ... }

end

end

Potential Pitfall: when sandboxing, I had an instance variable hash with symbol keys. When reading in the JSON file and writing to my instance variable, I generated an error in my methods… :key in original instance variable versus 'key' from incoming JSON.
 
Does the JSON file exist?

if FileTest.exists?('./file_name.json')

Read file

JSON.parse(File.read('./file_name.json'))

Open the file & write

File.open('./temp.json', 'w') do |f|

f.write(JSON.pretty_generate(object))

end

 
Reference:

https://github.com/lukekedz/ruby_json_template

http://ruby-doc.org/stdlib-1.9.3/libdoc/json/rdoc/JSON.html

http://ruby-doc.org/core-2.2.0/FileTest.html

https://stackoverflow.com/questions/5507512/how-to-write-to-a-json-file-in-the-correct-format/5507535

https://hackhands.com/ruby-read-json-file-hash/