Nokogiri - parsing multi-line `` tag as link and text

Argus9 Source

I am using Nokogiri to parse an RSS feed for a podcast. I am trying to grab a particular piece of data containing a link to the episode, so I'm using Nokogiri to parse the XML response for the RSS feed.

The relevant bit is below:

  <title>An awesome title!</title>

Nokogiri appears to be having a hard time grabbing the <link> tag though; I am able to get the <item> tag as a Nokogiri::Node object, and I can grab the title just fine with node.css('title').text, but when I try the same with node.css('link').text, I get a blank string.

I tried calling node.children.to_a to examine all of the children in this node, and I noticed something odd: the text inside the <link> tag is being parsed as a separate child:

[0] = {Nokogiri::XML::Element} <title>An awesome title!</title>\n
[1] = {Nokogiri::XML::Element} <link>
[2] = {Nokogiri::XML::Text}\n

Is there a way I can help Nokogiri properly parse this multi-line tag so that I can grab the text inside?

UPDATE: Here is the exact code I'm executing when I run into the issue.

require 'open-uri'
doc = Nokogiri::HTML(open('')) # Returns Nokogiri::HTML::Document
node = doc.css('//item').first # Returns Nokogiri::XML::Element
node.css('title').text # Returns "Abroad in Japan: Two weeks more in Japan!"
node.css('link').text # Returns ""
node.css('link').inner_text # Also returns "" - saw this elsewhere and thought I'd try it
node.children.to_a # Result, parsed by RubyMine for readability:

result = Array (14 elements)
 [0] = {Nokogiri::XML::Element} <title>Abroad in Japan: Two weeks more in Japan!</title>\n
 [1] = {Nokogiri::XML::Element} <subtitle>Chris and Pete return and they've planned out a very different route through Northern Japan.&amp;nbsp;\n\n\nOur Google Map can be found here:&amp;nbsp;\;nbsp;\n\n\nGet in touch:&amp;nbsp;[email protected]&amp;nbsp;\nMore Abr...</subtitle>
 [2] = {Nokogiri::XML::Element} <summary></summary>
 [3] = {Nokogiri::XML::Element} <guid ispermalink="false"></guid>
 [4] = {Nokogiri::XML::Element} <pubdate>Wed, 16 May 2018 21:00:00 GMT</pubdate>
 [5] = {Nokogiri::XML::Element} <duration>01:00:00</duration>
 [6] = {Nokogiri::XML::Element} <keywords></keywords>
 [7] = {Nokogiri::XML::Element} <explicit>no</explicit>
 [8] = {Nokogiri::XML::Element} <episodetype>full</episodetype>
 [9] = {Nokogiri::XML::Element} <image href=";w=1500&amp;"></image>
 [10] = {Nokogiri::XML::Element} <description>Chris and Pete return and they've planned out a very different route through Northern Japan. <p><br></p>\n<p>Our Google Map can be found here: </p>\n<p><a href="" target="_blank"> </a></p>\n<p><br></p>\n<p>Get in touch: <a href="mailto:[email protected]" target="_blank">[email protected]</a> </p>\n<p>More Abroad In Japan shows available below, do subscribe, rate and review us on iTunes, and please tell your friends! </p>\n<p><br></p>\n<p><a href="" target="_blank"></a></p>]]&gt;</description>
 [11] = {Nokogiri::XML::Element} <link>
 [12] = {Nokogiri::XML::Text}\n                
 [13] = {Nokogiri::XML::Element} <enclosure url="" length="28806528" type="audio/mpeg"></enclosure>

NOTE: One of the URLs above uses a URL shortener, which SO doesn't like, so I replaced it with



answered 5 months ago Casper #1

The fix is a lot simpler than you would think. An RSS feed is not valid HTML, but it works with XML:

doc = Nokogiri::XML(open('...'))

Ruby also has a module named RSS, which might be better suited for something like this:

require 'rss'
doc = RSS::Parser.parse(open('...'))
=> "https://...."

comments powered by Disqus