{"id":170,"date":"2015-05-14T04:09:20","date_gmt":"2015-05-14T04:09:20","guid":{"rendered":"https:\/\/existencia.org\/pro\/?p=170"},"modified":"2015-05-14T04:09:20","modified_gmt":"2015-05-14T04:09:20","slug":"google-scholar-alerts-to-rss-a-punctuated-equilibrium","status":"publish","type":"post","link":"https:\/\/existencia.org\/pro\/?p=170","title":{"rendered":"Google Scholar Alerts to RSS: A punctuated equilibrium"},"content":{"rendered":"

If you’re like me, you have a pile of Google Scholar Alerts<\/a> that you never manage to read. It’s a reflection of a more general problem: how do you find good articles, when there are so many articles to sift through? <\/p>\n

I’ve recently started using Sux0r<\/a>, a Bayesian filtering RSS feed reader. However, Google Scholar sends alerts to one’s email, and we’ll want to extract each paper as a separate RSS item.<\/p>\n

\"alertemail\"<\/p>\n

Here’s my process, and the steps for doing it yourself:<\/p>\n

Google Scholar Alerts → IFTTT → Blogger → Perl → DreamHost → RSS → Bayesian Reader<\/b><\/p>\n

    \n
  1. Create a Blogger blog that you will just use for Google Scholar Alerts: Go to the Blogger Home Page<\/a> and follow the steps under “New Blog”.<\/li>\n
  2. Sign up for IFTTT<\/a> (if you don’t already have an account), and create a new recipe to post emails from scholaralerts-noreply@google.com<\/tt> to your new blog. The channel for the trigger is your email system (Gmail for me); the trigger is “New email in inbox from…”; the channel for the action is Blogger; and the title and labels can be whatever you want as along as the body is “{{BodyPlain}}” (which includes HTML).\n

    \"ifttttrigger\"\n<\/li>\n

  3. Modify the Perl code below, pointing it to the front page of your new Blogger blog. It will return an RSS feed when called at the command line (perl scholar.pl<\/tt>).\n

    \"rssfeed\"\n<\/li>\n

  4. Upload the Perl script to your favorite server (mine, https:\/\/existencia.org\/<\/a>, is powered by DreamHost<\/a>.<\/li>\n
  5. Point your favorite RSS reader to the URL of the Perl script as an RSS feed, and wait as the Google Alerts come streaming in!<\/li>\n<\/ol>\n

    Here is the code for the Alert-Blogger-to-RSS Perl script. All you need to do is fill in the $url<\/tt> line below.<\/p>\n

    \r\n#!\/usr\/bin\/perl -w\r\nuse strict;\r\nuse CGI qw(:standard);\r\n\r\nuse XML::RSS; # Library for RSS generation\r\nuse LWP::Simple; # Library for web access\r\n\r\n# Download the first page from the blog\r\nmy $url = "http:\/\/mygooglealerts.blogspot.com\/"; ### <-- FILL IN HERE!\r\nmy $input = get($url);\r\nmy @lines = split \/\\n\/, $input;\r\n\r\n# Set up the RSS feed we will fill\r\nmy $rss = new XML::RSS(version => '2.0');\r\n$rss->channel(title => "Google Scholar Alerts");\r\n\r\n# Iterate through the lines of HTML\r\nmy $ii = 0;\r\nwhile ($ii < $#lines) {\r\n    my $line = $lines[$ii];\r\n    # Look for a <h3> starting the entry\r\n    if ($line !~ \/^<h3 style="font-weight:normal\/) {\r\n        $ii = ++$ii;\r\n        next;\r\n    }\r\n\r\n    # Extract the title and link\r\n    $line =~ \/<a href="([^"]+)"><font .*?>(.+)<\\\/font>\/;\r\n    my $title = $2;\r\n    my $link = $1;\r\n\r\n    # Extract the authors and publication information\r\n    my $line2 = $lines[$ii+1];\r\n    $line2 =~ \/<div><font .+?>([^<]+?) - (.*?, )?(\\d{4})\/;\r\n    my $authors = $1;\r\n    my $journal = (defined $2) ? $2 : '';\r\n    my $year = $3;\r\n\r\n    # Extract the snippets\r\n    my $line3 = $lines[$ii+2];\r\n    $line3 =~ \/<div><font .+?>(.+?)<br \\\/>\/;\r\n    my $content = $1;\r\n    for ($ii = $ii + 3; $ii < @lines; $ii++) {\r\n        my $linen = $lines[$ii];\r\n        # Are we done, or is there another line of snippets?\r\n        if ($linen =~ \/^(.+?)<\\\/font><\\\/div>\/) {\r\n            $content = $content . '<br \/>' . $1;\r\n            last;\r\n        } else {\r\n            $linen =~ \/^(.+?)<br \\\/>\/;\r\n            $content = $content . '<br \/>' . $1;\r\n        }\r\n    }\r\n    $ii = ++$ii;\r\n\r\n    # Use the title and publication for the RSS entry title\r\n    my $longtitle = "$title ($authors, $journal $year)";\r\n\r\n    # Add it to the RSS feed\r\n    $rss->add_item(title => $longtitle,\r\n                   link => $link,\r\n                   description => $content);\r\n        \r\n    $ii = ++$ii;\r\n}\r\n\r\n# Write out the RSS feed\r\nprint header('application\/xml+rss');\r\nprint $rss->as_string;\r\n<\/pre>\n
    In Sux0r, here are a couple of items form the final result:<\/p>\n
    \"sux0rfeed\"<\/p>\n","protected":false},"excerpt":{"rendered":"
    If you’re like me, you have a pile of Google Scholar Alerts that you never manage to read. It’s a reflection of a more general problem: how do you find good articles, when there are so many articles to sift through? I’ve recently started using Sux0r, a Bayesian filtering RSS feed reader. However, Google Scholar […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[3,4],"tags":[],"class_list":["post-170","post","type-post","status-publish","format-standard","hentry","category-research","category-software"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p4Zh9E-2K","_links":{"self":[{"href":"https:\/\/existencia.org\/pro\/index.php?rest_route=\/wp\/v2\/posts\/170","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/existencia.org\/pro\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/existencia.org\/pro\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/existencia.org\/pro\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/existencia.org\/pro\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=170"}],"version-history":[{"count":0,"href":"https:\/\/existencia.org\/pro\/index.php?rest_route=\/wp\/v2\/posts\/170\/revisions"}],"wp:attachment":[{"href":"https:\/\/existencia.org\/pro\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=170"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/existencia.org\/pro\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=170"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/existencia.org\/pro\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=170"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}