Recomended configuration for high users requests

[Darth Florus]

Mr. Farshad:

 

Our enterprise have the possibility to make (in conjuntion with the goberment) the software to catalog all the vehicles of Paraguay (about 3.000.000 of them). And we want to do it with UniGui.

 

We make some test using UniGui 0.93.0.996, Internet Information Server (via ISAPI) on Windows Server 2008 and Firebird 2.5.

 

All work very whell until we reach 100 - 110 concurrent users. After this all browsers request down!

 

We test algo with Apache and ISAPI module, but in this case only reach about of 60 - 70 concurrent users.

 

We need to reach almost 300 concurrent users (initialy).

 

My question is:

 

Waht configuration (hardware - software - platform) do You recomend me to make this application with UniGui?

 

Waht kind of information need I provide You to make me the recomendation?

 

I preciate very much some kind of tip!

 

Best Regards

[AlbertoVesx]

Maybe you are keeping in memory a lot information. Try to optimize that.

[Darth Florus]

The application used to try concurrent performance have not information on memory.

 

Only inserts rows on a table using a UniTimer to simulate user input.

 

I setup the timer to perform inserts between 30 and 60 secs.

[Darth Florus]

I have the same application on a Win32 environment and with it I can reach 300 concurrent users. Obviusly.

[Oliver Morsch]

Only inserts rows on a table

 

Do you use

(1) a database component such as TTable and use Table.append/insert

OR

(2) send SQL statements to the database server (insert into table ....)

 

2nd should be better for your purpose.

 

And how is the RAM usage on the server: How much it increases by a new user session and over time?

[Darth Florus]

Ja, ja, ja, TTable!. horror!

 

Nop I use a raw TSQLQuery (no ClientDataSets) with a parametrized Insert/SQL statement inside.

 

The server reach about 500MB and normally does not increment when user initiate new session.

 

Practically the server capacity is not used. The proccessor not pass over 30%. And the net card es about of 25% of mean.

 

I think is a configuration of IIS but I not find tunning information of this issue.

 

Some people says to me that ISAPI is a very bad and old technology election to use.

 

They recomends me ASP.NET.

[Farshad Mohajeri]

Well, well! What does the log file tell you?

[Farshad Mohajeri]

All work very whell until we reach 100 - 110 concurrent users. After this all browsers request down!

 

Is your app Win32 or Win64?

 

Have you checked the log file for errors?

[Jancarlos Martins]

Hi Oscar,

 

My suggestion, do not use TDataSource, creates class that manipulates the data,  like DAO in JAVA.

[Darth Florus]

All the logs files says "Session Timeout".

 

In the ServerModule:

 

Ajax Timeout: 60.000

Session Timeout: 600.000

 

Is Win64 and Win32 y try both of them.

 

I configure the IIS to execute WOW64\isapi.dll or System32\isapi.dll where is needed.

 

I try with IE, Firefox and Chrome. In differents computers.

 

The server managers says me that the Windows Server 2012 SP1 is the Standard Version.

[Darth Florus]

I don't use DataAware components in this project, I use TSQLQuery without ClientDataSet or TDataSource components.

 

This project only insert random rows on a tabla to test concurrency performance.

 

If I use a Bussiness Object class (like Java), internally I need to use a TSQLQuery is the same sittuation.

[Darth Florus]

Thanks for your opinions pals, I appreciate it!

[Amaury]

Be interesting to see the response of Farshad. I'm betting unigui, for construction of an application of high demand from users.

[Hayri ASLAN]

as you know we have car tracker software whose name is MyGPS.

 

at once we see 534 connected user.

 

our unigui software using max 70MB and %1 cpu.

 

i think you have a wrong thing.

[rencarnacion]

Hi, Hayri Aslan, what kind of tools are  you using to connect to the database, ADO, Firedac, BDE or DbExpress ?

Thanks

[adragan]

I have the same configuration (IIS + Firebird + ISAPI) and I run on an intranet app 80 users.

The processor rarely goes more than 10% on a not so great machine ( 2GB RAM , Win2008 ).

 

 

Here are some reasons Firebird could drive you to "Session timeout"

 

Firebird has the bad "habbit" to start garbage collection every 20000 or so transactions.

This is slowing down considerably the response of the server and so you can go "timeout" because of that.

You can choose a bigger no or disable "garbage collection" or set it on "manual"  when making lots of inserts.

 

Firebird keeps the indexies in sync after every transaction.

This is also slowing down the inserts but keeps everithing in the proper order , not as some very much

paised other SQL sistems.

Deactivating the indexies ( if it's an available option for the operation of the business) would make inserts a lot faster.

 

If you try to do 3 mil. inserts in a single tansaction the lock table grows too big and slow.

If you commit after every insert it also takes to much time.

Commiting after 500 - 1000 transactions seems to be an optimum.

On my laptop I noticed that inserts starts with 300 transactions/sec, then slows down gradually and

after some hundred thousend inserts it remains in a "serenity" start of doing nothing !!

That's the most probable reason for timeout.

 

You can  disable "forced writes" on the database if you have a UPS.

 

Try to creat a statement with parameters, prepare it and then execute the inserts.

 

TTable and such perform horribly on SQL servers in general.

Try to avoid them unless it's a table with max 10-20 records.

 

Check the above an see if you can improve someting.

[Darth Florus]

Thanks for Your recomendations!!!

 

To update the topic I find that the antivirus is slowing down network traffic. I add a port exception to it and then UPs to 250 concurrent user request!!!.

 

Its a shame that a Windows server need a antivirus to work securelly. On Linux servers that kind of issue does not exists.

 

In my case; when I put it in production environment; can't disable indexes because other types of users (police mans, hospitals, etc) need to check I a car have credentials to "rool over the street", at the same time of inserts occurs.

 

Because the inserts will be made from a lot of users every post must have its own transaction. Will be catastrofic if a row is not present on the table because are certifications of cars. I will care so much every row.

 

Forced writes is disabled because a UPS is present.

 

I use prepared queries with parameters.

 

The garbage collection recomendation is good I will try it!

 

A technician says me to use servers on a cluster, maybe I use it.

 

I see that use in service mode is bether than IIS + ISAPI, reacts more rapidly. So Is needed to implement a Guardian to start up when its down!

 

Best Regards

[chefdackel]

Waht configuration (hardware - software - platform) do You recomend me to make this application with UniGui?

 

Waht kind of information need I provide You to make me the recomendation?

 

 

you may consider separating the core logic from db access by implementing a n-tier architectur for db-access. So hosting the Firebird Server on another machine. At first glance it seems to be waste work and a waste of resources because of the additional layer, but you have many advantages:

 

.) separate the logic of the main program and the data server and data access, making the UniGui program more stable.

.) you can quickly change the database API - with minimal changes in the UniGui part.

.) you can host the data server on another machine and easily get the data with HTTP, Rest, Soap.... or whatever servers, not only by your UniGui program, but even by f.e. a windows desktop program which uses the same additional server to get his data.

.) security issues: if a hacker gets access to your UniGui windows server, he has no access to the real data server, which is on another machine. Its harder to steal the data.

 

take a look at:

 

.) Datasnap

.) kbmMW

.) RemObjects/DataAbstract

.) RealThinClient (RTC)

 

The downside is that it is not a set and done thing, you make take some days to get a real world application with all the bells and whistles and all the testing. Often documentation of these tools is outdated and not complete. But if you want a professional, high availability system than its worth to consider. You can go even a step further: why not taking all the printing logic also away from the core UniGui server and set up an additional server with one of the above tools for the printing part.

[Darth Florus]

Thanks ChefDackel for your recomendations.

 

To update the topic I says you the next:

 

There are a councilor of the UNDP (United Nations Development Program) involved on the project.

 

They says that we are to use ASP.NET with C#.

 

They develop Us the same example program maded on C# and we test it.

 

We reach at startup 3.000 concurrent user operations in the same server where we make our test with UniGui + ISAPI + IIS.

 

The server can support more users, but 3.000 is more than we need.

 

Obviusly the project manager decide to use ASP.NET with C#.

 

Its a shame, sorry pals, maybe next project!

 

Thanks for all your recomendation.

 

Best Regards

[adragan]

I know only too well the so called spetialists from audit companies.

They have no clue if  they are taken out from the Windows / Oracle world.

Linux is a mistery and the rest of the databases do not exist.

"Cover your ...."  seems to be their mission in life. 

[chefdackel]

They says that we are to use ASP.NET with C#.They develop Us the same example program maded on C# and we test it.

 

We reach at startup 3.000 concurrent user operations in the same server where we make our test with UniGui + ISAPI + IIS.

 

 

well, did they also use ExtJS in the ASP.NET example, f.e. use http://www.ext.net with ASP.NET? If not, its not a fair test. That would be an interesting comparison, build up some forms with some grids and input fields or whatever you want (both to compare alternatives based on ExtJS), use the same (external) DB server, and then do the testing. Maybe very exciting test cases.

[Darth Florus]

I know only too well the so called spetialists from audit companies.

They have no clue if  they are taken out from the Windows / Oracle world.

Linux is a mistery and the rest of the databases do not exist.

"Cover your ...."  seems to be their mission in life. 

 

Seems you know about that Mr. Adragan!. Thats all true!

[rsanford]

We really need to get uniGUI out of beta. Clients want to see full documentation and specifications. Several live production users/sites as reference is needed. I know that uniGUI will get there, but it has been a long road.