Get Full-Text Feed with Delphi

Question

I'm developing a Delphi program that reads the source of a feed through the component Indy idHTTP.. but the feed does not appear complete, appears only its summary, I mean the feed does not show the "content" tag.

I think it is possible to get full-text feeds because I have found some sites on the Internet (like http://fulltextrssfeed.com/) that can show the full-text feed, even for feeds that show only the summary.

What should I do to be able to read the full-text feed via Delphi? Is there a specific Delphi component to do this or can I do it with idHTTP?

Any help will be very very appreciated

Thanks you All!

Answer 1

1. Delphi has other built-in controls for TCP/IP besides Indy. Try one, and see if you get different results. Even as an experiment, it would only take about 30 seconds (drag drop component onto a form, set the URL, fetch the results, and see what you get). Correct?

2. If the site in question isn't giving you a full-text feed, I'm not aware of any Delphi component that will automagically get it for you. Here's a good link about some heuristics to get it yourself from the partial RSS contents:

   How to programmatically determine whether an RSS feed is a full feed or a partial feed

   And a tutorial on parsing the RSS, using Delphi:

   Reading and manipulating XML files with Delphi

3. Finally, there seem to be a number of web apps out there which CAN get the full-text RSS for you:

   http://fulltextrssfeed.com/feed.php

   http://www.fullrssfeed.com/

   http://www.unsum.com/converter

   Perhaps you can integrate one of these services into your app?

Answer 2

Seems that when RSS/Atom feed has no full-text content, the way to get it is to go into the feed's URL, and extract the full-text via innerHTML/outerHTML.. See below a good example:

PHP sources (AGPL v3 license): http://code.fivefilters.org/p/full-text-rss/source/tree/master/

I could do it on Delphi, though it will take some time to get done.. On the other hand, as Paulsm4 said above, it may be good integrate one of these services into your app.

So..well.. for now I'm thinking to provide both options(logic to get full-text, and integration) in my app. :)

Answer 3

For many web applications, the document returned from the web server can depend on additional headers in the HTTP GET request, for example the Accept header.

By default, Indy will set it to text/html, *.*.

Try setting it to application/rss+xml, the server then should reply with the full RSS content:

...
IdHTTP1.Request.Accept := 'application/rss+xml';
Response := IdHTTP1.Get(url);
...