Do you need an editor for fixed blocked files? 1

[agynamix]

Hi there,

I'm currently trying to do some research for a next software application I'm developing and I'm very interested in your opinion.

In nearly all of the projects I'm doing for customers (large Telcos) we need to handle files from host systems that are mostly fixed blocked.

Now it's very painful to edit/view/search those files in a standard text editor because you do not see the underlying record structure. I know for the host there is an application called "file aid" but for PC's I haven't found any (well Microfocus has such an editor in it's tool suite but it's old and very expensive).

My questions:

- Do you know any such editor that runs on Windows/Mac/Unix for an affordable price?
- Would you need such an editor?
- If you would be willing to buy such a tool what would be the features you'd like to see?
- Would you be willing to beta test?

Thanks for your help,

Torsten Uhlmann

 

[tcsbiz]

What does this have to do with COBOL?

 

[agynamix]

Well when I was a Cobol programmer we were readign and writing mostly fixed length files. I developed a lot of converters that would read in one format and spit out the other.

To test my application I had to feed it the proper files which I most of the time had to edit myself to implement certain test scenarios.

I know such an editor is not limited to Cobol developers but my guess was that many Cobol developers would work with flat files rather than XML. That's why I posted to this forum.

Torsten.

 

[webrabbit]

If your files are not too big, there is an old DOS-based editor that can handle fixed record length files. You do realize that on PC's "blocks" have no meaning? The name of the editor is SLED. It is shareware, but I have been unable to contact the author. I could send you a copy with documentation, if you desire. It is quite powerful for such a small editor.

 

[Glenn9999]

Hard to do better than XVI32 for a hex editor. Free, too.

Download from here:

http://www.chmaas.handshake.de/delphi/freeware/xvi32/xvi32.htm

Measurement is not management.

 

[agynamix]

I have search some more and found this editor:

http://record-editor.sourceforge.net/

which comes close to what I have in mind.

@webrabbit: What I meant are files of fixed length format, "fixed blocked" was a translation error I do though know that there are no block on PC systems.

 

[tcsbiz]

There's a ton of tools like this. I use DBU but you can find numerous of them, some free, some not.

What platform are you considering selling this for?

 

[agynamix]

Well, if this is an opportunity I'd develop this as an Eclipse (

http://www.eclipse.org)

application. I'd deliver it as an extension to an existing Eclipse installation for people that work with Eclipse and as a standalone application than can be used by itself with no knowledge of Eclipse internals.

The application would run on Windows, Mac, Linux and Solaris

It would have the following features:

- view/edit CSV and Fixed Length files
- export data into Excel or Html
- Search a file on a per-field basis
- Generate test data according to the record structure definition
- Compare two files on a per field basis
- And probably a whole lot more once people would start using it

 

[tcsbiz]

Another one I use I forgot to mention is Notepad++. It's an open source project.

 

[BruceMartin]

Ohers Editors you could look at are

* FillRED

http://www.want.ch/FillRed/

* jfFileAid

https://sourceforge.net/projects/jfileaid/

* FileAid has a PC version but it will cost a Lot though
* Some Comercial Cobol Compilers have an editors for there Files (i.e. MicroFocus)

At the moment the RecordEditor is most advanced of the 3 free editors. Apart from Generating Test Data it should meet your other requirements.

Rather than starting your own project, Why not work on one of the existing Projects. What you propose will take 2 to 3 years effort (see

http://www.ohloh.net/p/record-editor?ref=WidgetProjectThinBadge

it has an estimate of 15 man years for the RecordEditor code. This is an over estimate but it will take a lot of work).

 

[agynamix]

Thanks Bruce, for your comments. I greatly appreciate the effort you have put into Record-Editor!

A coworker in a recent project has found your software and he seems quite happy with it.

Do you know commercial alternatives? We would not use the Cobol Copybook integration because most of our projects are Java based on Unix servers (AIX, Solaris) applications that process files sent from host.

I've asked the original question to evaluate if there is a market for a commercial software and I found it best to ask those who work that field.

I'm an independent developer (well apart from the whole lot of time when I work for others ) and this project would be meant to pay my bills...

Thanks all for your insight!

@tcsbiz: I daily work with Notepad++ but how can I view a fixed length or CSV file in a per record way?

 

[BruceMartin]

My view is there should be a market for software like this on Unix / Windows, but there probably is not at the moment (not unless you did a lot of marketing any way). You need convince people change the way they do things.


When I wrote the RecordEditor, I was working for a company moving from the mainframe to Unix. The Ex-Mainframer’s used the RecordEditor and thought it was great; The Unix-C programmers would not have anything to do with it.

For me the big advantage of something like the RecordEditor in batch processing are:
1. You can split a large batch process into a lot of small programs linked together by files. These small programs should be easier to test and understand than having one large program. After all the Unix Shell system is based on piping files from one small program to another, so why not use the same idea.
2. The files can become your Log. If you have a problem check through the files unto you find the first file with the error. You then know which program has the error.
3. You can sort the files into DB sequence this can reduce DB processing
- If the DB is Clustered on the Index
- If you can reduce the number of DB calls
4. Supports Binary files. Binary files can have a much lower overhead than Text, CSV or XML.


Example of using files (instead of DB’s):

Sales-To Inventory Transaction Processing
- On the Mainframe (2 processors 8 gb ram) running a one thread - 20 minutes processing
- On Unix C / Oracle the initial version (processing 0.1% of the data on 16 processor box) ran for 8 hours. After a lot of optimization, several hardware upgrades (to 32 Processor IBM Regata) changing the program to run multiple threads they where able to run the sales processing in around 10 hours.

If you do decide to write something, Have a look at jrecord.sf.net.
1. It is LGPL license so can be redistributed
2. Holds the RecordEditor’s IO and Record Access routines
3. There is also ByteReader’s / ByteWriters these read/write on logical record from a file as an array of Bytes. In the next version (coming soon) there will be version for
* Fixed Length Records
* 3 different VB files (basically a line consists of a Record Length followed by the Data
* Standard Windows / Unix files.

Good luck with it

Bruce

 

[agynamix]

Bruce,

thanks for your detailed response!

Do you know any commercial applications that do not come with the big Cobol packages?

I have spoken to a few people in my current organization:

- operators that need to monitor our applications. There are times when they need to search through these files to find an occurence of a certain token/date/etc.

- testers that need to edit files to create their testcases. Also in an automated test scenario a tool would come handy that can compare files based on the record definition and only compare certain fields/record types.

- spec writers. I'm not quite sure what they need it for but one of the people writing the spec found your tool.

I also think that there is a market I just cannot quite grasp it yet.

 

[BruceMartin]

I do not know of any Commercial offerings apart from those related to Cobol or the mainframe (File-Aid (compuware), File Master (CA ?) and IBM did have something as well). Going Back 20 years, there quite a number of similar programs on the Mainframe, but the smaller companies got swallowed up leaving the Big players.

At one company I worked for the Cics Front-End programs only did Screen handling; the business logic / DB access was all done in Backend programs. For Testing the backend we would setup the Test data with FileAid. Write a program like


Read Input Test
While not end-of-file
Call Backend
Write Backends-Response to output file
Read Input Test
End-While

While there is nothing pretty about this process, Good points include
· Test runs are much Faster and more reliable than some Screen recording programs that I have used.
· For regression testing can just compare the files
· If input to the backend changes, you copy the Test file and add the extra Fields with FileAid. With Screen recorder’s you may have to rerecord the script.

Wether this would be a good technique in the java world I do not know

 

[Crox]

I work with SPF/SE from Command Technology Corporation, an editor that understands the editing commands from the IBM mainframe, but it can do a lot more. For example: you can execute change commands or even macro's on selected sources. You can for example select sources by searching for something inside the source and change only those selected sources. I use it all the time also for conversions... (i have no shares or so, just a very satisfied user)

 

[mikrom]

Hi Crox,

But SPF/SE is an source code editor, not a database editor.

 

[Crox]

i thought it was about "Do you need an editor for fixed blocked files?" ....

 

[agynamix]

Crox: Yes I was actually looking for editors that can be used to edit fixed length files. Thanks for the pointer!

 

[Wjburke2]

I have been programming for about ten years and also use SPF/SE on the PC. It is a great fixed length file editor. Especially for handling editing Mainframe files on the PC. It is not just for source code but it is good in that area also. Liked it so much I bought my own copy.

 

[webrabbit]

In the past, I used SPF/PC on the PC. I found it to be clumsy to use as it emulated the mainframe terminal processing logic too closely. Is SPF/SE better in this regard? It sounds like it may be.