Delete duplicates from text file

[kre1973]

I have the following text file below thats has duplicates records in it, in which I need the duplicates removed thus creating a new text file....

"8002220365archwireless.net"|Baker, Scott
"8002220365archwireless.net"|Baker, Scott
"8002257514archwireless.net"|Boston, Jeff
"8002257514archwireless.net"|Boston, Jeff
"8009410781archwireless.net"|Cannon, Ryan
"8009410781archwireless.net"|Cannon, Ryan
"8002225412archwireless.net"|Davis, Savitha
"8002225412archwireless.net"|Davis, Savitha
"8009411510archwireless.net"|Donn, William
"8009411510archwireless.net"|Donn, William
"8002220587archwireless.net"|Evans, Dan
"8002220587archwireless.net"|Evans, Dan
"8002010113archwireless.net"|Fox, Anthony
"8002010113archwireless.net"|Fox, Anthony


New Text file below

"8002220365archwireless.net"|Baker, Scott
"8002257514archwireless.net"|Boston, Jeff
"8009410781archwireless.net"|Cannon, Ryan
"8002225412archwireless.net"|Davis, Savitha
"8009411510archwireless.net"|Donn, William
"8002220587archwireless.net"|Evans, Dan
"8002010113archwireless.net"|Fox, Anthony


thanks

 

[woogoo]

Hi, I'd create a custom class to store the data in and a collection to store your class in :

Public Class Person
    Private m_URL As String
    Private m_Name As String

    Public Sub New()
        m_URL = Nothing
        m_Name = Nothing
    End Sub

    Public Sub New(ByVal Title As String, ByVal Name As String)
        m_URL = Title
        m_Name = Name
    End Sub

    Public Overrides Function ToString() As String
        Return m_URL
    End Function

    Public Property URL() As String
        Get
            Return m_URL
        End Get
        Set(ByVal value As String)
            m_URL = value
        End Set
    End Property

    Public Property Name() As String
        Get
            Return m_Name
        End Get
        Set(ByVal value As String)
            m_Name = value
        End Set
    End Property
End Class

Public Class People
    Inherits CollectionBase

    Public Sub New()
    End Sub

    Default Public Property Item(ByVal Index As Integer) As Person
        Get
            Return (CType(List(Index), Person))
        End Get
        Set(ByVal value As Person)
            List(Index) = value
        End Set
    End Property

    Public Function Add(ByVal Value As Person) As Integer
        Return List.Add(Value)
    End Function

    Public Function IndexOf(ByVal Value As Person) As Integer
        Return List.IndexOf(Value)
    End Function

    Public Sub Insert(ByVal Index As Integer, ByVal Value As Person)
        List.Insert(Index, Value)
    End Sub

    Public Sub Remove(ByVal Value As Person)
        List.Remove(Value)
    End Sub

    Public Function Contains(ByVal Value As Person) As Boolean
        Return List.Contains(Value)
    End Function

    Public Function Find(ByVal S As String) As Boolean
        Dim P As Person = Nothing, bFound as False
        For Each P In Me
            If (P.Name.SubString(0, 10) = S) Then
		bFound = True
                Exit For
            End If
        Next
        Return bFound
    End Function

End Class

Create an instance of the Collection Class:

Dim MyPeopleCollection as People = new People()

Then read the file in a line at a time and grab the numeric portion of the URL and pass it to the find method of your collection.

If (NOT MyPeopleCollection.Find(URL)) Then
    MyPeopleCollection.Add(new Person(FullURL, Name))
End If

If you don't find it then add that entry to the collection, if it is found then ignore it. By adding a few lines of code to the collection class you can have it write out a new file for you with the, now unique, entries it it.

The code shown here won't run without some tweaking and adding the appropriate Imports but you get the idea.

Hope this helps.