Creating a macro in Visual Studio.NET

[tmcneil]

All,

I'm trying to write a macro in Visual Studio.NET to place INSERT statements around a data dictionary. I'm familiar with creating simple ones, but not sophisticated ones. I'd like to populate an Oracle table with this data for future use. A small sample of the data dictionary looks like this:

a
A
AA
AAA
Aachen/M
aardvark/SM
Aaren/M
Aarhus/M
Aarika/M
Aaron/M
AB
aback
abacus/SM
abaft
Abagael/M
Abagail/M
abalone/SM
abandoner/M
abandon/LGDRS
abandonment/SM
abase/LGDSR
abasement/S
abaser/M
abashed/UY
abashment/MS
abash/SDLG
abate/DSRLG
abated/U
abatement/MS
abater/M
abattoir/SM
Abba/M
Abbe/M
abbé/S
abbess/SM
Abbey/M
abbey/MS
Abbie/M
Abbi/M
Abbot/M
abbot/MS
Abbott/M
abbr
abbrev

As you can see, the file contains inconsistent data. Some have "/" at the end followed by some upper case letters, so I would like to remove the "/" and everything to the right of it. Some words have nothing after it, so that'll make it a little more difficult. Not sure why that is, but I'd like to wrap this around each word:

INSERT INTO EN_US (EN_US_WORD)
VALUES ('abacus');

This file has over 62,000 lines in it. it might be better to write a program to parse the data and wrap the statements around it. I'll do that if it is a last resort. Any ideas?

Todd

 

[cpjust]

Macros (in C++) are ugly neo-templates from the old C era. I'm sure you don't want to use a macro. You can do what you need with a simple function that reads the data from your file one line at a time, parses out the / and everything after it, and writes your INSERT statement to another file.

You can also do this in VBScript or any number of other languages. Are you familiar enough with C++ or other languages to read/write files, and parse strings?

 

[tmcneil]

cpjust,

I'm taking a c++ class and I'm actually trying to write a program to do just that. I'm opening the file and trying to read the data. I've declared an infile and I can open the file successfully. I have declared a char word [50] to read each word from the file. Not sure if this is right, but when I step through it I can see each word populating the char array. I've also used a string word and I can see that populating the string. I'm trying to print out the values so that I know that I am reading the words correctly. I have this so far:

string word;
int count = 50;

infile >> word;
while (!infile.eof() && count < 50)
{
    infile >> word;
}

printing is not working correctly
for (int i=0; i < count; i++)
    cout << i+1 << word(i) << endl;

I guess I could do the processing inside the while loop or create methods that I call to do the work.

Todd

 

[uolj]

You can also do this in VBScript or any number of other languages.

I think Todd was referring to a VB macro using VC++'s Macro facilities, not a C/C++ macro.

To do a single line, I would record a quick macro that starts at the beginning of the line, then uses Ctrl-Shift-RightArrow to select the word. I would then Ctrl-C to copy the word, then go back to the beginning of the line, Shift-End to select the whole line, and then delete it. I would then type "INSERT INTO EN_US (EN_US_WORD)" hit enter and type, "VALUES ('" then Ctrl-V to paste the word, then type "');" and hit RightArrow to move the cursor to the start of the next line. Then I'd end the recording.

You could play this recorded macro 62,000 times to do the whole file, or you could edit a new macro, then open up the TemporaryQuickMacro and copy the code it created and combine it with a loop to run 62,000 times (or however many you want).

 

[cpjust]

string doesn't have an operator() defined, so

word(i)

won't compile.

Replacing the (i) with will just give you the character at position i in the string. To save all the strings that you pull out of the file, use a vector<string> instead of just a string. You can then add the words to the vector like this:

#include <iostream>
#include <iterator>
#include <fstream>
#include <string>
#include <vector>

using namespace std;

int main()
{
	// Open the file.
	ifstream infile( "c:/temp/test.txt" );

	// Read file into the vector.
	istream_iterator<string> begin( infile );
	istream_iterator<string> end;
	vector<string> words( begin, end );

	// Print all the words.
	for ( vector<string>::iterator it = words.begin(); it != words.end(); ++it )
	{
		cout << *it << endl;
	}

	return 0;
}

 

[tmcneil]

cpjust,

I spent some time after I was able to concentrate and get something written that will do what I need it to do. I am able to open the file, process each word by stripping out the "/" and all characters after, add a sinqle quote, "'", next to another one if it exists in the word, then output the processed words with the SQL insert statements wrapped around it. Here's the code:

//*****************************************************************************
// This constructor prints out header information to the screen.  Opens infile
// for input and outfile for output.
//*****************************************************************************
dict_proc::dict_proc(void)
{
    cout << "\n\n\t\t  Data Dictionary Processing Program" << endl;
    cout << DASHES;
    cout << "\n\t\t\t         by";
	cout << "\n\t\t    Programmer Name: " << NAME << endl;
    cout << DASHES << endl << endl;
    cout << "\n\n\t\t    Processing Data Dictionary...." << endl << endl;

    //open dictfile for reading
    dictfile.open ("en_US.dic", ios::in);
    if (dictfile.fail()) //see if file opened successfully
    {
        cout << "File, en_US.dic, could not be opened!\n\n";
	    system ("pause");   //so the message can be read
	    exit (1);  // abort the program
	} // end if

    //open restfile for reading
    restfile.open ("restrict.dic", ios::in);
    if (restfile.fail()) //see if file opened successfully
    {
        cout << "File, restrict.dic, could not be opened!\n\n";
	    system ("pause");   //so the message can be read
	    exit (1);  // abort the program
	} // end if

    //open good_file for writing
    outfile.open ("good_file.txt", ios::out);
    if (outfile.fail()) //see if file opened successfully
    {
        cout << "Error Opening File, good_file.txt!\n\n";
        system ("pause");
        exit (1);
    } // end if
} // end constructor

//*****************************************************************************
// process_input_file(): method opens a file, en_US.dic, and reads words into a
// string for processing.  Once processed, writes word to a file.
//*****************************************************************************
void dict_proc::process_input_file(void)
{
    //int wcount = 0, rcount = 0;
    string one_word, rword, word;
    string insert = "INSERT INTO EN_US (EN_US_WORD)";
    string values = "VALUES ('";
    string end    = "');";

    /*restfile >> rword;
    while (!restfile.eof())
    {        
        cout << rword << endl;
        //call validate_word()
        //word = validate_word (words);

        //write record to good_file
        //outfile << insert << endl << values << word << end << endl;

        restfile >> rword;
    } //end while*/

	dictfile >> one_word;
    while (!dictfile.eof())
    {        
        //call validate_word()
        word = validate_word (one_word);

        //write record to good_file
        outfile << insert << endl << values << word << end << endl;
        
        dictfile >> one_word;
    } //end while

} //end process_input_file

//*****************************************************************************
// validate_word(): method recieves word string, ie. "Aaron/M".  Checks to if
// a "/" character exists, if it does, remove that character and any other
// characters thereafter.
// Returns processed word string.
//*****************************************************************************
string dict_proc::validate_word (string word)
{    
    int pos;
    int apos;

    //find position in word where "/" exists
    pos = word.find("/");

    if (pos > 0)
        //strip out "/" and all chars thereafter using substr
        word = word.substr(0, pos);

    //find position in word where "'" exists
    apos = word.find("'");
    
    if (apos > 0)
        //Replaces a single apostrophe character beginning at position apos
        //with the characters "''".  Oracle does not recognize the "'"
        //character.
        word = word.replace(apos,1,"''");

    return word;
} // end validate_word

There is one problem. The end result of creating a data dictionary is to use it for a profanity filter that I am building. I have to go through this large file and remove any profanity words from it that I find. One thing I do have, is a restricted words file that I could use to compare it to. If they match, then remove that word from the data dictionary output. So, what's the best approach? Create an array to store each string and then do the string compare?

Todd

 

[cpjust]

Are you just removing 'bad' words everywhere, or are you keeping a score of how bad the word is, and removing it if it exceeds a certain threshold?
For example, some people might think "breast" should be removed. But what about "chicken breast" or "breast cancer"? If the word is used those ways, you probably don't want to remove it...

 

[tmcneil]

Remove only the word "breast" from the data dictionary and not any words that have breast embedded in it.

 

[tmcneil]

Well, I'm using an array to store each word so that I can compare it to the words in the restricted words list. I'm doing a "find", "substr" and a "replace" on each word by storing them into a string prior to processing. So, now that I have a processed word, how do I store it back into the array? my array is declared as follows:

char en_words[MAX_ELEMS][30];

process_words() method:

void dict_proc::process_words (void)
{    
    int pos;
    string insert = "INSERT INTO EN_US (EN_US_WORD)";
    string values = "VALUES ('";
    string end    = "');";

    for(int i=0; i < wcount; i++)
    {
        //read char into string to process
        str_en_word = en_words[i];
        //find position in word where "/" exists
        pos = str_en_word.find("/");

        //strip out "/" and all chars thereafter using substr
        if (pos > 0)
            str_en_word = str_en_word.substr(0, pos);
            //cout << pos << endl;

        //find position in word where "'" exists
        pos = str_en_word.find("'");
    
        if (pos > 0)
        //Replaces a single apostrophe character beginning at position apos
        //with the characters "''".  Oracle does not recognize the "'"
        //character.
        str_en_word = str_en_word.replace(pos,1,"''");

        //en_words[i] = str_en_word;  //cannot convert from 'std::string' to 'char[30]'

        //write record to good_file
        //outfile << insert << endl << values << str_en_word << end << endl;
    }

} // end process_words

 

[cpjust]

Instead of using a

char en_words[MAX_ELEMS][30];

why not use:

vector<string> en_words;

Then you get all the benefits of the string member functions and safety.

 

[tmcneil]

cpjust,

Well, I don't know? I'm not very knowledgeable about vectors. Can I use the same code that I have written and just replace the array with the vector? I'll try a few things and see how it goes.

Thanks for the tip.

Todd

 

[cpjust]

I'm not sure what the code looks like where you declare and populate the en_words array, but you'd probably need to make some changes for it to work with a vector.

A vector is a class that acts like an array that automatically grows to the size you need.

To add a string to the end of the vector, do this:

vector<string> en_words;

en_words.push_back( "first-word" );
en_words.push_back( "second-word" );
...

You can access the elements of the vector with the [] operator just like an array, or you can use an iterator.