Comparing files not working as intended

Question

hi guys could someone explain to me why this does not work.

I basically have to text files called Books and NewBooks...

The text files are populated from a web request and the info is then parsed into the text files...when I start the program Books and new books are identical and pretty much a copy of each other.

more web requests are done to update the NewBooks text file and when I compare them if there is a line in NewBooks that is not in Books it adds that line to a third text file called myNewBooks. Now my initial code that I will show here works as I expected

Dim InitialBooks = File.ReadAllLines("Books.json")
    Dim TW As System.IO.TextWriter
    'Create a Text file and load it into the TextWriter 
    TW = System.IO.File.CreateText("myNewBooks.JSON")

    Dim NewBooks = String.Empty
    Using reader = New StreamReader("NewBooks.json")
        Do Until reader.EndOfStream
            Dim current = reader.ReadLine
            If Not InitialBooks.Contains(current) Then 
                NewBooks = current & Environment.NewLine

                TW.WriteLine(NewBooks)
                TW.Flush()
                'Close the File 
            End If
        Loop
    End Using
    TW.Close() : TW.Dispose()

but because part of the string in my text file lines contain a url which sometimes I find the same book with a different url... I was getting duplicate entries of books becuase the url was the only difference. So I thought that I would split the string before the url so that I just compare the title and description and region ...fyi a line in my text files look similar to this:

{ "Title": "My Title Here", "Description": "My Description Here", "Region": "My Region Here", "Url": "My Url Here", "Image": "My Image Here" };

So a fellow today helped me figure out how to split my line so it looks more like this:

{ "Title": "My Title Here", "Description": "My Description Here", "Region": "My Region Here", "Url"

which is great but now when I compare it does not see that the first line contains the split line and I don't understand why... here is the code after it was modified.

Dim InitialBooks = File.ReadAllLines("Books.json")
    Dim TW As System.IO.TextWriter
    'Create a Text file and load it into the TextWriter 
    TW = System.IO.File.CreateText("myNewBooks.JSON")

    Dim NewBooks = String.Empty
    Using reader = New StreamReader("NewBooks.json")
        Do Until reader.EndOfStream
            Dim current = reader.ReadLine
            Dim splitAt As String = """Url"""
            Dim index As Integer = current.IndexOf(splitAt)
            Dim output As String = current.Substring(0, index + splitAt.Length)
            If Not InitialBooks.Contains(output) Then 
                NewBooks = current & Environment.NewLine

                TW.WriteLine(NewBooks)
                TW.Flush()
                'Close the File 
            End If
        Loop
    End Using
    TW.Close() : TW.Dispose()

Your wisdom would be appreciated!!

Answer 1

Your OP is confusing.

If I understood correctly:

You have 3 files Books, NewBooks and MyBooks. You download data from web, if that data is not located in Books, you add it to NewBooks, otherwise to MyBooks(duplicates).

Seeing that you are working with JSON i would do it the following way.

Load the Books, when downloading data check it and compare it with Books. Then write to proper file.

Imports System.Web.Script.Serialization ' for reading of JSON (+add the reference to System.Web.Extensions library)
Dim JSONBooks = New JavaScriptSerializer().DeserializeObject(Books_string)

Inspect JSONBooks with breakpoint. You will see how it looks. When downlaoding data you can simply check if book exist in it, by title, url or whatever you want.

Since you shown only one book

Debug.Print(JSONBooks("Title")) 'returns >>>My Title Here

When you have more

JSONBooks(x)("Title") 'where x is book number. 

So you can loop over all books and check what you need.

JSON array looks like this (if you need to construct it)

[{book1},{book2},...]