TOPIC:

Took me so much time to go from mysql to postgresql, because of my previous topic and now this!
To be honest - it's a facepalm already. ZeOS DBO components leak in a simple select
Here's my setup:
CT 5.5 (32-bit BIG IDE) on Windows 8.1 x64
PostgreSQL server: ver. 9.4 on Windows 8.1 x64; ver. 9.2 on Windows XP Pro
ZeOS version: 7.2.0-beta (in component version field), 7.2.0.22 according to ctinfo.txtA set of TZConnection and TZQuery dropped on a form, connect to Postgre, ZQuery performs a single select on a non-empty table.
Turn on heap-trc before compiling project and look at the result.
Here's my report in the image attached.

Tomorrow I'll try to downgrade to ver. 7.1 stable and will report

Sir
here is pl_ZEOS 7.1.2 stable
from CT 4.7
Please test and report

Works like a charm with the version you, Mr. Sternas, provided.
Test made with PostgreSQL server v. 9.4 (x86) on Windows 8.1 x64, selected almost all kinds of fields, including bytea. No memory leaks detected from my test project. Will recompile my main project and will report further.
Regarding my previous thread - the mysql connections in threads - it's the same as before - exceptions thrown when freeing connections, i.e. nothing different compared to v. 7.2.0.22

Sir
we are waiting for your final opinion (7.1.x or 7.2.x)
it's critical for us

So far ver 7.1.2 (sync 19-12-2013 SVN Rev 3025 ) works with PostgreSQL in threads (tried with 25 simultaneous connections, a simple select of a large table), with most of column types, stored procedures and functions and I didn't find anything wrong.
Regarding mysql - seems to work in a single thread, haven't noticed anything unusual in component behavior. Currently don't have any other servers installed, as I uninstalled MariaDB, so cannot confirm whether it works or not, but if it's working with mysql, it should work with MariaDB also.
If I find anything wrong, I'll post here ASAP.
For those, who are gonna revert from 7.2.0.22 to 7.1.2 - don't forget to Clean up and Build your projects (menu Run -> Clean up and Build...)!

OK
we will continue with 7.2.x in Lab CT 5.6 revision 005510 (ETA I hope soon...)

Hi Sternas,

the reported issue contiues.
Current ZeosDBO is in huge trouble with FPC3.x gen. Actually i can't say why. The line the "Javor" reports was a simply System.SetString(). And indeed millions of memory leaks are reported if i'm trying to use CT 5.5/6.

BUT it's not a problem of Zeos, rolling back to a deprecated 2.7/6 eveything is fine. Since a week i'm trying to make simple project for the FPC core to raise a ticket.... but i fail. Inbetween i've the idea there is soemthing broken using global functions + !!! Interfaces !!!
So the report of Javor is correct but NOT a ZeosDBO issue.. I need some help in this domain.

The differences between Zeos-7.1 and Zeos-7.2 is:
In 7.1 series everything was a String or UnicodeString which means for the Unicode IDE's like D2009+ or FPC in delpiunicode mode it was a nop. Same types. So i've been spitting this processing to faster RawByteString or UnicodeString (according driver dependencies). But the codepage agnostic idea of FPC3+ seems to be broken for some unknown reasons.

Michael, ZeosDevTeam

Michael Hiergeist wrote: Hi Sternas,

the reported issue contiues.
Current ZeosDBO is in huge trouble with FPC3.x gen. Actually i can't say why. The line the "Javor" reports was a simply System.SetString(). And indeed millions of memory leaks are reported if i'm trying to use CT 5.5/6.

BUT it's not a problem of Zeos, rolling back to a deprecated 2.7/6 eveything is fine. Since a week i'm trying to make simple project for the FPC core to raise a ticket.... but i fail. Inbetween i've the idea there is soemthing broken using global functions + !!! Interfaces !!!
So the report of Javor is correct but NOT a ZeosDBO issue.. I need some help in this domain.

The differences between Zeos-7.1 and Zeos-7.2 is:
In 7.1 series everything was a String or UnicodeString which means for the Unicode IDE's like D2009+ or FPC in delpiunicode mode it was a nop. Same types. So i've been spitting this processing to faster RawByteString or UnicodeString (according driver dependencies). But the codepage agnostic idea of FPC3+ seems to be broken for some unknown reasons.

Michael, ZeosDevTeam

"The Javor" reported what he saw. The Javor tried to all versions (before the post) of ZeosDBO 7.2.x and there was a problem. The Javor downgraded back to 7.1.stable and everything worked like a charm. The Javor noticed the difference: 7.1.x worked fine, 7.2.x leaked. System.SetString was a no concern to The Javor. Please, read, try and then report here the errors I've spent too much time to find your errors. It's not only in Zeos package. I use CT for serious projects with a huge load and not just a DB control with a single table small things. Yes, FPC has its problems, but logic says the following: a version of FPC with 7.2.x = problem; the very same version of FPC with 7.1.stable = no problem. Nothing further to say

Guys, we want to help

My suggestion ,
someone to post here a sample project with the problem(s)
and we will solve all problems in any level, FPC or Zeos.
Our lab and 5+15 programmers waiting...

For us, ZEOS is Extreme Critical library

@Javor

please stay professional. I didn't say you'r report is wrong. I'd say the line you've reported is inside ZCore function ZSetString was a system.SetString. Indeed i've been tracking the issue replacing SetString by SetLength ... the issue is gone. Therefore i was forced to introduce a define {$WITH_SETSTRING_MEMLEAK_BUG} for FPC >= 3.0 which reduces the memory leaks a loads.

It's not my intention to say you did something or your report is wrong nope, the more i'm informing CT there is a huge problem i actually am not able to find the origin. And the issue was introduced since ??? (sorry for my absence) i did upgrade to CT 5.4/5. Since i was failing to find out what's wrong with this revision i simply was waiting for the next CT version. Horray CT5.6 was released .. sadly i've to announce the weired issue continues.

That's why i wrote: I need help. And that's why i pointed the differences out.: 7.2 uses the correct RawByteString whereas 7.1 uses the String in loads of the functions. 7.2 is loads faster intitially because of this but as long the weired memory-leaks do continue... broken arrow. There is not one memoryleak with FPC2.6 or any Delphi release but millions (if i run my ZTestAll -> SVN) which happen since a for me unknown version of FPC3.x.

OK, Jervas? Your report is ok! Realy it is! And others should know it too.
I'm forced to announce there are some/millions of more such memory-leaks. I can't judje if other projects (except mORMot which is in same trouble using the RawByteString as well) have same issues in this domain. Current FPC3 is a non useable target for me. But 7.2 want's going to stable, even since 7.3 is already in production (new direct OleDB/ODBC drivers for FPC too) but i can't set the attribute as long we're in such a weired trouble with latest FPC compilers.

@Sternas,
make simple project using ZeosDBO. First add the heaptrc unit in first place to your project, Connect to a database like Postgre, MySQL, SQLite, FireBird what ever you want. Fetch some data of a simple table. Or make some updates it doesn't matter. Close your app .... read the memory reports. I can guarantee the is not ONE with FPC 2.6, 2.7 or Delphi but with all latest CT used FPC targets. If you want just comment my define in ZeosLazarus.inc and the are a millions more reported line.

A FPC hero is wanted. We need a simple project for the FPC Bugtracker. They are not willing to download the ZeosDBO sources. Therefore since one week i'm trying to mak a little project to reproduce the issue ... puff no mem-leak.

OK
now the problem it's very clear.
I will stop all LAB running tasks and from tomorrow we will focus only on this
(we using the latest ZEOS from SVN trunk)
Give us time,
I will report any progress here

PS: can someone test Lab CT 5.70 revision 005641 ? info

@Michael
Thanks for the info provided. That may narrow searches. Bad part my side is that currently 3 of my projects (relying on CT + Zeos + lNet basically) need to go alive again. No time to trace errors, so far I can confirm that my CT LAB ver. along with its Zeos ver (can provide Mon, since it's on my work PC) seems to go, more or less, fine. No mem leaks spotted so far on both MySQL and PostgreSQL DBs. lNet needs some tweaks, but that's another point. Tried even my sample project provided with MySQL where leaks were detected in earlier versions - it seems to be OK so far. Just simultaneous connections in threads (MySQL DB) seem to disconnect much slower than expected, yet they do w/o any exception raised. PostgreSQL part seems to work good too. Don't get me wrong, I want to help, I just don't have the time to dig in deeper. My job (projects) are depending on this stuff...

@Javor,

well it wasn't my intention to blame you or to be rude. Just note i'm main constributor os ZeosDBO. Doing the job for free (no donations) is enough. My latest refactorings pushes the Zeos performace in a very high rate (something about factor x10 dependend to driver, read the synopse.info docs) an for free. I do understand you and others. But same happens to current Zeos project, we're in queue for a stable FPC/Delphi behavior. Trunk should be merged against \testing-7.3 since a long time but this issue is a longstanding problem i can't handle by my selves. So each help is welcome, same as your report.

But guiding by writing how to do .... to much for me. -> Family, job etc.

@sternas

going to bed now. I'll give it a shoot tomorrow morning @4:00 a clock. An check what happens.

Good night

@Javor
You are very clear Sir
I understand your situation, NO problem

We will make our test tomorrow, with half-Team Lab configuration ...

@Michael
I must go home too, but I am waiting to finish lab CT Building on Scientific Linux 7.2 MultiArch

sweet dreams ...

Good morning Sternas.

Night was short and it took a while until i found out how to use the developer version.
After disabling my bullguard-antivirus i got it run.

First approach to run the ZeosTests did fail with a sigsev see attached picture. Note MDeams (Zeos project-manager) did also report same error last days. AFAIK he uses the FPCUP, i just use CT.
This error happens just on startup. I can't set a breakpoint somwhere.

Trying to make a little tast app now and see what happens...

Good morning Sir
I am alone here, oh, with my first coffee
Updating ZEOS to SVN trunk Rev 3815

any test project ? to start testing ?

Good morning Sternas, Sir.

Got my second coffee inbetween. Actually i make a little app. It will take some more minutes but if i'm ready i'll post again.

OK, we have a lot of time..

New findings, Sir

The sigsev occours on using the heaptrc unit. Comment the unit in the project and you're able to get it running.

Which than means ... i can't judge if something has been changed inbetween
Maybe you can find out what's going on? Is there a difference because of the huge memorymanager refactorings you guys did on ct5.7?
Edit: Nope i don't think so else MDeams wouldn't report same sigsev -> Next FPC issue

Just try it by your selves and give me a sign..

Morning gentlemen. Glad to see great efforts are made towards problem solving. We're kinda on the same boat after all. My pain is that after every update of CT many new things happen with my projects and sometimes a very strange behavior occurs. Some of my projects are servers, heavily using Zeos, lNet, etc. and they have to be on 24/7. Sometimes, when I have time, I take deeper look inside, trying to find out what causes problems. So far, bugs found are all reported, including some in lNet, but they were reported back to Almindor (and confirmed later) for further tests. My point was that back then, in my Delphi projects, I tried many DB components, including free and shareware, some of which used a direct TCP connection to the servers, i.e. no libraries needed, and all worked fine even in many-threads-project (using both MySQL and PostgreSQL, tried also FB and SQLite back then).
So updating whole CT along with rebuilding IDE and toolchains, which cost me half of a day or just some components, without tests passed is unacceptable for me. When I get to the office tomorrow, I'll do my best to get the latest CT on one of my VMs and try to compile sample posted here and my old samples to check status. If I do it, I'll report here.
Have a good day