I had a lot of fun learning to scrape, because I saw the potential to use that to automate some of my online activities. I decided to create a scrape of a blog I like to check every week (when a certain column I like is usually published). With that, I wanted the scraped data (generated into HTML) to be emailed to me once a week at a certain time (after the column is published).
The first step requires the Ruby Mail Gem. It’s pretty simple. First:
gem install mail
Then, at the top of my scrape program I required it:
Next, within my program, I set the mail options, which I figured out from the gem documentation and some googling about gmail configurations:
Then, I set the mail defaults, which sets the method of deliver to
:smtp and my set
1 2 3
After that, I entered the delivery options. As far as I could tell, you can only send from one gmail from another, not any other email server. Additionally, because gmail is particular, if I send an email to myself externally like this, it won’t show up in the inbox, but the sent messages. To bypass this, I sent the email from my junk email address that I keep for purposes such as this. In the subject line, I used string interpolation to make the subject be the title of the article. In the
text_part, I set the body to say something like “this email is not supported in plain text”. This message will never be seen in gmail; it’s for mail clients that don’t support HTML. The real body of the email is in the
html_part. I set the
content_type to ‘text/html; charset=UTF-8’, and the body the interpolation of the article content, from my scrape.
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Running my program from the command line, it sends the email!
I then wanted to automate this even further by getting the email sent to me only once a week, at a certain time. Some research lead me to a feature within Mac OSX that automatically runs programs upon certain parameters. This is the same feature that automatically launches certain programs upon rebooting the machine. It’s called launchd or launchctl.
It requires making a
.plist file in
~/Library/LaunchAgents. When I went into this folder, I found a file already in there that launches Spotify upon rebooting (mystery solved on that). Before deleting that file, I copied the template to make my file,
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
The important part of this was setting
StartCalendarInterval to the day, hour, and minute. This tells the computer to run the program (in the
ProgramArguments) when indicated. Within the
ProgramArguments I indicated the location of ruby, and the location of my program.
To test to make sure the
.plist file works, not just at the time I indicated, I ran this:
launchctl start com.myname.myprogram
Waiting in my inbox moments later was my newsletter. :)