amanda himmelstoss

I had a lot of fun learning to scrape, because I saw the potential to use that to automate some of my online activities. I decided to create a scrape of a blog I like to check every week (when a certain column I like is usually published). With that, I wanted the scraped data (generated into HTML) to be emailed to me once a week at a certain time (after the column is published).

The first step requires the Ruby Mail Gem. It’s pretty simple. First:

gem install mail

Then, at the top of my scrape program I required it:

1

require 'mail'

Next, within my program, I set the mail options, which I figured out from the gem documentation and some googling about gmail configurations:

1
2

mail_options = { :address => "gmail-smtp-in.l.google.com",
                 :port => 25 }

Then, I set the mail defaults, which sets the method of deliver to :smtp and my set mail_options above:

1
2
3

Mail.defaults do
  delivery_method :smtp, mail_options
end

After that, I entered the delivery options. As far as I could tell, you can only send from one gmail from another, not any other email server. Additionally, because gmail is particular, if I send an email to myself externally like this, it won’t show up in the inbox, but the sent messages. To bypass this, I sent the email from my junk email address that I keep for purposes such as this. In the subject line, I used string interpolation to make the subject be the title of the article. In the text_part, I set the body to say something like “this email is not supported in plain text”. This message will never be seen in gmail; it’s for mail clients that don’t support HTML. The real body of the email is in the html_part. I set the content_type to ‘text/html; charset=UTF-8’, and the body the interpolation of the article content, from my scrape.

1
2
3
4
5
6
7
8
9
10
11
12
13
14

Mail.deliver do
  to 'my email address'
  from 'my junk email address'
  subject "#{article_title}"

  text_part do
    body "this email is not supported in plain text"
  end

  html_part do
    content_type 'text/html; charset=UTF-8'
    body "#{article_itself}"
  end
end

Running my program from the command line, it sends the email!

I then wanted to automate this even further by getting the email sent to me only once a week, at a certain time. Some research lead me to a feature within Mac OSX that automatically runs programs upon certain parameters. This is the same feature that automatically launches certain programs upon rebooting the machine. It’s called launchd or launchctl.

I figured this out, again, through some googling. This helpful comment on Stack Overflow gives a good explanation on how to use Launchd to run jobs.

It requires making a .plist file in ~/Library/LaunchAgents. When I went into this folder, I found a file already in there that launches Spotify upon rebooting (mystery solved on that). Before deleting that file, I copied the template to make my file, com.myname.myprogram.plist.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
 <key>Label</key>
 <string>com.myname.myprogram</string>
 <key>StartCalendarInterval</key>
 <dict>
  <key>Hour</key>
  <integer>12</integer>
  <key>Minute</key>
  <integer>0</integer>
  <key>Weekday</key>
  <integer>4</integer>
 </dict>
 <key>ProgramArguments</key>
 <array>
  <string>/Users/username/.rvm/rubies/ruby-1.9.3-p448/bin/ruby</string>
  <string>scrape program's location</string>
 </array>
</dict>
</plist>

The important part of this was setting StartCalendarInterval to the day, hour, and minute. This tells the computer to run the program (in the ProgramArguments) when indicated. Within the ProgramArguments I indicated the location of ruby, and the location of my program.

To test to make sure the .plist file works, not just at the time I indicated, I ran this: launchctl start com.myname.myprogram

Waiting in my inbox moments later was my newsletter. :)

Before starting the Flatiron prework I had never been inside my terminal window before. I knew what it was, but I never ventured in, mostly out of fear of breaking something. I knew terminal was serious techie territory; navigating around on the computer without it was just using an interface, and those who knew the terminal were just computer geniuses. I however, wasn’t that. I have always been good with computers, comfortable and capable of general problem solving. I mean, I even knew what terminal was; not many I know do.

While doing the prework and going over command line basics, my mind was in many ways blown. A simple -ls while in a directory I knew very well from the GUI side of things suddenly had a whole new display. Navigating through directories, making (and deleting, but very carefully) files from the command line offered me a sense of control over my machine* and an understanding of what it all looked like, really.

One thing I have been grappling with through the prework and in the first days of school is the fear of breaking things. Through the command line, I’m in a different world than what I’ve always been familiar with. And this world feels more serious. Before, I could move a file through dragging it, and I wouldn’t lose it or get confused, because I myself physically put it there. From the command line, things could get confusing if I’m not careful or paying attention. Before I could delete a file of folder and it would go safely into the trash bin. Now, a simple rm -f or rm rf deletes it forever. That’s serious.

Even with code, I’ve paused before running it not out of anticipation of what it might do, but with what might break, even if it’s just testing. I’m used to living in a world where I can’t really break my computer, because I’m just on the surface. The command line changes this. One of the most important things with programming is being comfortable with breaking things, and I’m trying every day to have more courage in doing that.

*Once you start referring to your pretty MBP Retina as a machine I think means you’ve crossed over to the nerd side of things :)

When beginning to learn Ruby, I never came across RSpec in the list of buzzwords and acronyms associated with programming. With my first exposure to RSpec and the concept of Test Driven Development, I was a bit confused about what the purpose was. When writing methods, I assumed calling methods gave all of the testing necessary to know that your program works. RSpec, I’ve learned, goes beyond that.

Jason Arhart’s RSpec for Beginners presentation for the Las Vegas Ruby Group givens a concise yet comprehensive overview of what the purpose of using RSpec tests is. The point that really stuck with me is the idea of “focus on the what before the how”. One of the things I’m making a conscious effort to practice while learning to code is to slow down, think methodically, and break down each necessarly step. Writing RSpec tests is one way to do this. In order to write an effective RSpec test, you must know first what you’re program needs to do. This must be done before the how<.

Another thing Jason focuses on in his presentation is how RSpec tests account of expected behavior. We talk about this sometimes in class, and it’s one thing I like to think about when programming: it’s important to account for all the things a user might do with your program. I like to think about these things while programming because one of the most exciting things about coding for me is the idea of my products being used by others and how they use them; RSpec testing creates a comprehensive way of testing for all of those different user possibilities.