Design Advice 2

[VB400]

Background:

In an n-tier application, the data-centric tier is a dll running inside of MTS. This dll has many classes that are considered the other half of the ui-centric business objects. So in the ui-centric tier, we may have a business object called customer while the data-tier counterpart is called customerpersist. The latter provides methods for customer retrieval and update and so on. Everything works great!

The issue:

Now, we're at the point where we need to provide reports. Regardless of the reporting package (Crystal Reports, Active Reports, etc.) we need to standardize our reporting process within the n-tier model.

The options (please add to this list):

1) Add more methods to existing business objects to retrieve all the data needed for reports. For example, we may have a data-tier business object that has methods such as Fetch, Save, GetDataForReport1, GetDataForReport2, etc. For some reason, I feel that this option would dilute the effectiveness of the original design. Meaning that we would be creating bigger and bigger objects that do too many things.

2) Create new objects that are designed specifically for desired reports. I feel that this is more inline with good component design. For example, we can create a separate dll that has business objects that are designed for reporting. Now, if we have users (say management) that look at data without ever updating it, all we would need is the dll for reporting.

Any thoughts, opinions, advise is highly welcomed.
Tarek

The more I learn, the more I need to learn!

 

[Brice]

Since our app is relatively small in scope, what we've decided to use is single reports class with a method for each report on the ui-centric side and on the server side or more specifically, data-related business rules side, a single reports module with either one generic/all-purpose method that accepts a parameter array and calls a third-party reporting package or a method for each report that does the same thing (we haven't decided yet).

We're still in the early stages of coding it all so I'm still a little unfamiliar with the third-party package, but it looks promising so far.

 

[chiph]

Tarek -

Since people wanting reports seem to change their requirements on a whim, I'd add a new "Reports" layer above your Persist layer. That way you wouldn't be shipping new core components eight times a week.

It also allows you to hide some of your data. Why give internal-only fields to the reporting people, who will use them for purposes they weren't designed for? (happened at my last job - customer didn't use the API we published, and when we changed how the internals worked, they pitched a fit).

BTW, what reporting package are you going to be using? Last job was just starting with a beta copy of Crystal Reports that spit out .pdf files. The new job uses a high-dollar package called Business Objects.

Chip H.

 

[VB400]

Brice,

Thanks for your input -- creating a single class with multiple methods (one per report) was my first thought as well. However, I'm now curious about Chip's thought regarding this extra layer above the Persist layer.

Chip,

I agree with you regarding the users changing their minds on a whim. However, I'm not so sure what you mean by this new layer. Are you talking about a new DLL within the MTS package or are you talking about a new class within the existing DLL or perhaps a totally separate MTS package?

In any event, I'm not sure how this would help me not ship new components every week. For example, say we create an object that runs *somewhere* on the application server machine and all it does is receive requests for report data and it returns that data to the caller. If the users now ask for the report to be modified, I would have to make changes to that new back-end-object. Additionally, I would have to ship the modified reports to all the users.

It seems that I'm missing something here. BTW, this is not a web application -- it uses forms and the UI-Centric object exists on each user's machine.

As for the reporting package, for now, we're using the most basic Crystal Reports package that came with VB6. I'm sure as time goes on, we'll have to upgrade to a more robust app but this will do for now.

Thanks!
Tarek

The more I learn, the more I need to learn!

 

[Brice]

I admit, I'm a little curious too. Our situation is such that the reporting requirements are fairly static, so we shouldn't have to ship new stuff all that often. Also, the application has a fairly small number of users, so upgrades shouldn't be too much of a hassle when we do have to do them.

 

[chiph]

Tarek -

Sorry it's taken me so long to reply -- I started the new job on Monday, and they got me involved in stuff right away.

Yes, I meant a new DLL (aka MTS package, aka COM+ package). My idea was that you'd create a DLL that make a certain set of data available for reporting purposes. It would talk to your "core" DLLs. If someone needed slightly different data, you'd create a new DLL for their requirements, and it too would talk to the core DLLs.

The costs/benefits in this approach are:
1) Flexibility
2) Users see only what you want them to see
3) You don't have to modify existing components every time a new report comes out
4) Performance hit since you're now going through two layers.
5) You end up with a lot of components.

I suspect #4 above won't be so bad, since MTS and the operating system will cache your DLLs. In conjunction with #5, you'll want to have a good naming scheme in place so that you can tell the DLLs apart.

Chip H.

 

[VB400]

Chip,

Here's what's on mind so far:

1. I build new dll that will be used in a new MTS package on the application server. This dll will have one or more classes to return the desired data for a report. If the report requirements are changed, say we need another field added to this report, we change the &quot;Reporting&quot; dll and we do not effect the core application dll (the one currently used for editing and updating data). If this is what you meant then what did you mean by <It would talk to your &quot;core&quot; DLLs>?

2. The Fetch function in the new DLL should accept one parameter that is basically a property bag. This is so that if the report requirements change, the interface doesn't change. The request itself is included within the serialized property bag.
Tarek

The more I learn, the more I need to learn!

 

[chiph]

If the report requirements are changed, say we need another field added to this report, we change the &quot;Reporting&quot; dll and we do not effect the core application dll (the one currently used for editing and updating data). If this is what you meant then what did you mean by <It would talk to your &quot;core&quot; DLLs>?

Errrr, that's exactly what I meant. You'd have a set of core application DLLs that all they do is fetch, create, update, delete. You then write reporting DLLs above them that present the info the report needs.

The Fetch function in the new DLL should accept one parameter that is basically a property bag. This is so that if the report requirements change, the interface doesn't change.

Have you thought about using XML for this?

Chip H.

 

[VB400]

Chip,

My MTS package currently has one dll that has all of the data-centric objects in it. If I understand you correctly, I could (maybe even should) break down my existing dll to multiple dlls and run them all in the same package. I could break them down by functionality such as by application module. If I continue with this thought then I could add a new dll for reporting to this package. Is this correct? Is this what you and most others do?

The next question would be this: say one of the components in one of the dlls in the package needs to be replaced. Can I, while the package is in use, replace the component that needs updating? I'm assuming of course that at that time, none of the components within the dll are in use (otherwise we wouldn't be able to replace the dll in the folder to begin with).

As far as XML is concerned, does it work with NT/4.0 or is it strictly in Win/2k?
Tarek

The more I learn, the more I need to learn!

 

[VB400]

I just tested my theory and it didn't work. I was able to include many dlls in the same package and it works well.

I created two front end programs -- one program uses one of the components in one of the dlls while the other program uses a component from the other dll.

However, I decided to make a change to one of the classes in one of the dlls (code only -- no interface changes). So, I deleted the component from the package while the package is running -- no problem. I then tried to replace the dll with Windows Explorer and I got a &quot;file in use&quot; type of a message. This is because (I assume) although nothing from the dll is currently being used, the dll itself is locked by MTS or Windows or somthing.

Thus, my theory (hope) that I can have multiple dlls and at any time I can replace any of the dlls in a pakcage while the package is running is thrown out the window.

Unfortunately, I cannot use multiple packages because the database connections that I'm using are a bit prohibitive unlike the SQL Server connections.
Tarek

The more I learn, the more I need to learn!

 

[jcasetnl]

Seems like the second idea is better. In addition to the reasons you mentioned it also has the added benefit of being easier to secure. You could put them in seperate apps and adjust security appropriately. Without this you're relying primarily on obsurity to protect the report functionality.

best ..
- j

 

[Snaggs]

Tarek,

XML is a way to describe your data. It looks a little like HTML in the idea that it has opening and closing tags. However, you get to define the tags yourself. The nice part about it, is that the tags can be in any order and they don't all have to be there. You can add new ones without breaking the application. Here's a stripped down version of one that we used to use.

<Customer>
<FirstName></FirstName>
<LastName></LastName>
<Address1></Address1>
<Address2></Address2>
<City></City>
<State></State>
<Zipcode></Zipcode>
</Customer>

When you fill it with data, it looks like this:

<Customer>
<FirstName>John</FirstName>
<LastName>Doe</LastName>
<Address1>1212 Main St.</Address1>
<Address2>This tag can be left out if there's no data for it</Address2>
<City>Phoenix</City>
<State>AZ</State>
<Zipcode>85134</Zipcode>
</Customer>

This method works real good wrapped in a class. The application I wrote that did this used a socket to pass all the information back and fourth between the tiers.
Snaggs
tribesaddict@swbell.net
Enter any 11-digit prime number to continue...

 

[chiph]

VB400 -

I'm out of suggestions then. I didn't know that your DB connections were so limited (you must be using Oracle at $140 a seat).

Chip H.

 

[VB400]

Snaggs,

Is XML a feature of Win/2K or is it available for NT/4.0? We're currently using NT/4.0 and developing in VB6.

Chip,

Thanks for your help, your earlier suggestions already helped!
Tarek

The more I learn, the more I need to learn!

 

[Snaggs]

VB400,

It'll work for your environment. From the book VB6 XML from Wrox, pg 14.

There is nothing in the XML specification that favors one platform over another. XML is pure text, which make it a platform neutral data format. Just to be clear on this, the specification states that allowable characters are tab, carriage return, line feed and the legal graphic characters of Unicode and ISO/IEC 10646. This simplicity is essential when trying to exchange data between any two arbitrary systems. It has tremendous value even when all of your applications are on the same platform.

Another part of the book that more closely talks about what you're asking about goes as follows. pg 16.

A large software company has different applications in various languages, that run on different platforms. Some are in C, VB, Java, etc... As they continue development, they begin to see that some desired features already exist in some form or another in an existing program. To fully make use of their investment, they decide to make the existing functionality openly available by exposing XML interfaces (meaning that the public methods take XML strings as parameters, and return XML string as the response). So they modify the existing applications to allow transactions via socket requests. An application could listen on a given port, waiting for a connection, and when one comes in, XML could be passed in, the request handled, and XML sent back out. New applications could now request services from any of the existing applications, regardless of the platform.

I wanted to use the verbiage that Wrox had in their book because they explained it to a tee! This is basically the environment that I created for the application that we redesigned. Here was the setup:

Windows NT Server running SQL Server. On the same machine was a custom DLL written in C++ that used with the Microsoft XML parser also known as the DOM (Document Object Model). The DLL made a connection to a SQL Server database when it was initialized and then waited for a transaction. Since this was a DLL, there had to be an application to launch the DLL. I wrote the server side in Visual BASIC and added a reference to the C++ DLL in the server side application. I refer to the Server part I wrote as a BROKER. All it did was open a Service Port on the server and waited for a client to connect to it via a socket. When a connection came in, it was accepted by the VB application. When data came in on the socket, the XML was passed from the VB application to a method in the C++ DLL. The C++ DLL had all the logic in it to deal with the requests. This was done for speed. Once the DLL was done with processing the request, it spit back a string (an XML string) that was just passed back to the client application. When the client received the XML data, it disconnected from server. So the VB Application that ran server side was only passing data back and fourth and had no logic in it what-so-ever.

The VB application that ran server side would make only one connection to SQL Server (thus saving on SQL Server licenses), but the intent was to charge for connections to our server (VB Server).

There was also a client side DLL that was written in Visual BASIC. I wrapped all the DOM calls and XML calls into the DLL so it was totally transparent to the user of the COM object. Basically all you had to do to program the client side DLL was set a reference to it and call a method. The DLL took care of all the connecting, disconnecting and parsing of the XML. In fact if I hadn't told you it was XML based, you would never know it.

In short, a socket is like a pipe that you open up between two machines that allows bi-directional communications over a network using TCP/IP for the protocol. You run the VB server application, it instantiates a C++ DLL and makes a connection to SQL Server. At that point the server is waiting for a connection.

You run the client application and the user does something that calls the client side DLL. Properties are set on the object and then a method is called. Inside the client DLL, the properties of the COM object are picked off and placed into the DOM. The DOM then creates and XML string with Elements and Tags in it. Then the client application makes a connection to the server, passes the XML string up the socket and waits for a response.

The VB server receives the response, takes the XML string and passes it to a method of the C++ DLL and it waits for a response. The VB server then takes the XML string from the C++ DLL and passes it back down the socket to the client.

The client receives the XML string from the server and loads it into the DOM for parsing. The values are pulled off the DOM and placed into properties of the object on the client side. Then the client issues a socket disconnect.

At that point it's ready for another transaction. This may seem like a lot, and it is, but on 95% of the transactions you'd never know the request went across the network and back.

The nice thing about this approach is that if you publish your XML routines and how to use them you could write applications that run on any platform, in any language. The socket part is nice because you can run the server anywhere in the world. Or the client as a matter of fact.

We had a guy on site that was having a problem with the client application. He called me up and I asked him for the IP. He gave it to me. I hooked the server 4 states away, right from my desktop. I was able to debug the application in the VB IDE and be connected to the data on a server 4 states away. Talk about an awesome development environment for debugging! That was sweet. I found the problem, and in 5 minutes I had e-mailed a new EXE to him. As with so many applications, it was a problem with data.

So there’s the long version of how XML was used in an application. There is an XML forum here that’s really good. A lot of it is over my head, but that’d be the place to ask questions.

Hope that helps. Snaggs
tribesaddict@swbell.net
Enter any 11-digit prime number to continue...

 

[VB400]

Wow!

As my signature says -- The more I learn, the more I need to learn!

So it seems that DOM (XML Parser) is the key to using XML and it must exist on the client and server in order to make and break the XML string. From the looks of it, it is a Microsoft product.

I guess my best bet is to look into the book you referenced &quot;VB6 XML&quot; and see how I can get started step by step.

Thanks for all your help Snaggs!
Tarek

The more I learn, the more I need to learn!

 

[Snaggs]

VB400,

Go out to this site and download the code from the book.

http://www.wrox.com/books/Book_Details.asp?isbn=1861003412§ion=&amp;subject_id=&amp;subject=

Get the code for &quot;Beginning XML&quot; from the site. Look at &quot;peopleattr.xml&quot; in the zip file.

The nice thing about this is you can just double click on the files from in WinZip and IE 5+ will load them. Look at all the examples in the file to get a feel for how data is represented in XML.

Some of the files have to have the XSL sheet to go with them. You might be better off unzipping all the files to a directory and then using Windows Explorer to look at them. Be careful cause some of them use an XSL style sheet and you'll be surprised by the output. Some of if looks really nice! Snaggs
tribesaddict@swbell.net
2 wire mesh butchering gloves:
1 5-finger, 1 3-finger, pair: $15

 

[chiph]

Here's some sample XML VB6 code that I used to serialize a class.

This first bit turns my class members into XML
[tt]
Public Property Get XMLPayload() as String
Dim objDOM As DOMDocument30
Dim objRoot As IXMLDOMElement
Dim objChild As IXMLDOMElement

Set objDOM = New DOMDocument30

Set objRoot = objDOM.createElement(ELEMENT_ROOT)
Call objDOM.appendChild(objRoot)

Set objChild = objDOM.createElement(ELEMENT_CLIENTID)
objChild.Text = Str(m_lClientID)
Call objRoot.appendChild(objChild)

Set objChild = objDOM.createElement(ELEMENT_XMLFILENAME)
objChild.Text = m_sXMLFileName
Call objRoot.appendChild(objChild)

XMLPayload = objDOM.xml

Set objChild = Nothing
Set objRoot = Nothing
Set objDOM = Nothing
End Property
[/tt]

The next bit turns a XML stream back into my class members:
[tt]
Public Property Let XMLPayload(RHS As String)
Dim objDOM As DOMDocument30
Dim objNodeList As IXMLDOMNodeList

Set objDOM = New DOMDocument30
objDOM.loadXML (RHS)

Set objNodeList = objDOM.getElementsByTagName(ELEMENT_CLIENTID)
m_lClientID = GetNodeTextLong(objNodeList)

Set objNodeList = objDOM.getElementsByTagName(ELEMENT_XMLFILENAME)
m_sXMLFileName = GetNodeTextString(objNodeList)

Set objNodeList = Nothing
Set objDOM = Nothing
End Property
[/tt]

Which uses a couple of helper functions I wrote:
[tt]
Private Function GetNodeTextString(ByRef TheNodeList As IXMLDOMNodeList) As String
Dim objNode As IXMLDOMNode

If TheNodeList Is Nothing Then
GetNodeTextString = &quot;&quot;
Else
Set objNode = TheNodeList.Item(0)
If objNode Is Nothing Then
GetNodeTextString = &quot;&quot;
Else
GetNodeTextString = TheNodeList.Item(0).Text
End If
End If
End Function

Private Function GetNodeTextLong(ByRef TheNodeList As IXMLDOMNodeList) As Long
Dim objNode As IXMLDOMNode

If TheNodeList Is Nothing Then
GetNodeTextLong = 0
Else
Set objNode = TheNodeList.Item(0)
If objNode Is Nothing Then
GetNodeTextLong = 0
Else
If IsNumeric(objNode.Text) Then
GetNodeTextLong = Val(objNode.Text)
End If
End If
End If
End Function
[/tt]


Hope this gets you started.

Chip H.

 

[VB400]

Chip,

What file did you have to reference to do the declaration:

Dim objDOM As DOMDocument30

Tarek

 

[chiph]

I set a reference to MSXML2, which is available at Microsoft (msdn.microsoft.com/xml). Look for XML Parser 3.0 (also get the SDK so you get the docs for it).

The docs aren't the best in the world, but since XML is changing so fast, they're not bad. DOM is much better documented than SAX.

The difference between DOM and SAX, is that in DOM, it loads the entire document into memory, and you can then navigate the tree. In SAX, it reads the file bit by bit, and raises events as it identifies parts of the XML document (startDocument, startElement, characters, endElement, endDocument are the ones used most often).

We're using SAX for our import routines, since we've got no control over the size of the files (A client could upload 20k one day, and 2gb the next).

Chip H.