A few months ago I began to play around with Ticketmaster.ie, Yahoo Pipes and Twitterfeed. I thought I’d share the process so you can have a go at mashing up your own feeds.

1. Ticketmaster

First you need a mash-up that’s going to be of some use, I chose Ticketmaster.ie as they don’t have a presence on Twitter and I could see how valuable it would be to hear of gig announcements in my Twitter timeline.

I googled Ticketmaster RSS feeds and came across Justin Mason‘s blog entry in which he details how to set up a RSS feed for a particular venue in Ireland, you can read more at:

Justin Mason's Blog

Justin compiled a small script, tm2rss.cgi, which scrapes the venue pages and produces an RSS feed.

For venues, simply take the venue URL (for example, http://www.ticketmaster.ie/venue/198641 for The Village), add the numeric venue ID in place of NNNNN in this URL: http://taint.org/scraped/tm2rss.cgi?v=NNNNN, then use that as the Feed URL in your feed reader.

In this example I will detail how I made the Twitter feed for Whelan’s venue in Dublin. The URL for Ticketmaster’s Whelan’s venue is
http://www.ticketmaster.ie/Whelans-tickets-Dublin/venue/198293 so to get the RSS feed I need to use Justin’s script changing the ‘N’ values to the venue number —
http://taint.org/scraped/tm2rss.cgi?v=198293.

We now have an RSS feed for Whelan’s:

Whelan's RSS Feed

We will need to restructure the feed into something more meaningful and that’s where Yahoo Pipes come in.

2. Yahoo Pipes

Yahoo Pipes is a very powerful web application that provides a graphical user interface for building data mashups that aggregate web feeds, web pages, and other services, creating Web-based apps from various sources, and publishing those apps.

There are some great resources and tutorials available for anyone wanting to experiment with its features. The best documentation to get started is the Pipes Troubleshooting manual provided by Yahoo.

Go to Yahoo Pipes and click on MyPipes, this will take you to an editing page where you can create your first pipe. (At first it may look a little daunting but it soon begins to make sense.)

Fetch Feed

Click on ‘Sources’ and drag ‘Fetch Feed’ on to the editing area. Add your URL to the input field, in this case we will be adding the feed for Whelan’s we created earlier — http://taint.org/scraped/tm2rss.cgi?v=198293. This should now looks as follows:

Fetch Feed

If the Fetch Feed module is selected you’ll notice in the Debugger window at the bottom of the screen the ‘title of the RSS feed’ is one item — Thu 03/12/09 20:00 – Cass McCombs:

Title Feed

We will want to split this item into a title and a date so we can restructure the feed; to do this we use the ‘Rename’ operator.

Rename

We now use the Rename function found under ‘Operators’. Again drag this on to the editable canvas and link the Fetch Feed module to it. Use ‘item.title’ and ‘Copy As’ and call the first field ‘tm_date’ and the second ‘tm_artist’:

Rename Feed

You will now notice that we have 2 new fields in the output:

New Items in Feed

Now the fun starts, we have to restructure the RSS items using the Regex operator.

Regex

Before attempting to write a regular expression I’d recommend reading the Pipes documentation on Regex. We want to have a Date and Artist’s item so for the first input we will input ‘item.tm_date’ and we will want to replace this item with our restructured item.

Regex

The regular expression to get the date is:

^(\w{3})\s+(\d{2})/(\d{2})/(\d{2})\s+(\d{2}):(\d{2})\s+-.+

This looks complicated at first but when it’s broken down it’s actually quite self-explanatory.

  • The “^”, when it’s the first character in a regex, says that what follows it should be matched at the beginning of the text we’re searching.
  • Next, we come to (\w) which is a Character class that represents “any word character”.
  • The “{3}” modifies whatever precedes it. In this case “\w{3}” means “exactly four word characters”
  • Then there is a (\s), which is another character class that represents “any whitespace character” so this simply accounts for the space after the day of the week.
  • The plus sign following “\s” modifies whatever it follows: “+” means “one or more”. So “\s+” means “one or more whitespace character”.
  • The (\d) represents “any digit” so this will take care of the first 2 digits of the date (day) using {2}.
  • The forward slashes (/) between each (\d{2}) shows that we know what this character is and therefore directly select it.
  • The period character is special. It’s a character class that means “any character”, so “.+” means “match one or more of any character at all”.
  • The hyphen at the end of the expression is another special character that we know will always come between the date and the artist.

In the ‘Width’ field we enter $2/$3/$4. For every matched parenthesis in the regex, a numbered reference is created. That reference, preceded by a dollar sign, can be placed in the rule’s value field. In our case, we have six sets of parenthesis, and we want to select the day, month and year for the date, separated by forward slashes, so we enter $2/$3/$4.

The item.tm_artist is exactly the same with different expressions attached to the end:

^(\w{3})\s+(\d{2})/(\d{2})/(\d{2})\s+(\d{2}):(\d{2})\s+-\s((?=.).{1,60})+

The \s((?=.).{1,60})+ simply specifies that the artist/band should be a string between 1 and 60 characters.

Here’s a handy Regular Expressions cheat sheet [PDF] that will help you get to grips with Regex.

Create RSS

Now that the dirty work is over we now have to create the RSS. Here we specify the Title, Description, Link and Publication Date as follows:

Create RSS

We have used item.tm_artist for the Title of the RSS, item.tm_date for the Description, item.link for the Link and item-pubDate for the PubDate.

Pipe Output

If we now connect Create RSS and Pipe Output we should get the following in the Debugger:

Output

The whole Pipe can be viewed here:

Yahoo Pipes Example

3. Twitter Feed

Now that we have our working Pipe we want this to be published to our Twitter account (in this case in @ticketmaster_ie‘s timeline) and this is where Twitterfeed comes in. Twitterfeed is a tool for linking and publishing RSS feeds to a Twitter account.

First of all a Twitterfeed account is needed so sign up, then create a feed. Choose where you want to publish the feed to, either Twitter or Facebook, then authenticate your username and password. The settings for the rest of the page are as follows:

Twitterfeed - Whelan's

Copy the RSS feed from your Yahoo Pipes and paste it into the RSS Feed URL field and test it, it should come up as “Feed parsed OK, but due to missing pubDates make sure you select ‘Post new items based on GUID’ in Advanced Settings below.”

I’ve selected for Twitterfeed to check for new posts every 30 minutes and post up to 5 new items at a time. The post content I have chosen is ‘Include: title & description’. Be sure to uncheck ‘Feed is sorted’ for this example as we don’t want Twittefeed to be checking the whole RSS feed every half hour, just the new items that appear at the top of the feed. The ‘Post Suffix’ simply appends the name of the venue to the end of the tweet, the final output takes the following format:

Tweet Output

And that’s how you create a Yahoo Pipes and Twitter mash-up.

Twitter Mash-ups

Never miss your favourite band or artist in Ireland again with Ticketmaster_ie — Follow me here.

Ticketmaster IE Twitter

Keep up-to-date with all the music in Ireland with MusicIreland — Follow me here.

Music Ireland

You might also like these posts: