Forum - Support
joinus

Online

No members online
Members: 0 - Guests: 97

Newsletter

warnerp warnerp OFFLINE
Post : 12

stuck on backup

#1085 20 Dec 2012 01:05
I just installed this on a test machine to try this out and it seems to be stuck. First it sat on "categories backup" with the circular progress bar moving for about 1.5 hours. I hit the browser refresh and "create the import package" again, but then it got stuck on "medias backup". I refresh it again and it goes through and gets stuck on "creation du package". I've tried this multiple times with both Chrome and Firefox and it now gets stuck on "creation du package" each time. I've waited about 10 minutes thus far.

Are there dependencies I should be aware of or a log file I can check to see what the hang up might be or am I just not being patient enough with this?

Update--- I just checked the apache access logs. It's returning a 500 error code each time it gets to step 15.
Last Edit: 20 Dec 2012 01:12 by warnerp.
netassopro NetAssoPro OFFLINE
Post : 615

Re: stuck on backup

#1087 20 Dec 2012 09:49
Hi,
Can look after javascript error during the process, maybe you have a bad return and so the script go down.
warnerp warnerp OFFLINE
Post : 12

Re: stuck on backup

#1109 08 Jan 2013 20:09
I did some deeper investigation into this. Looks like it might be 2 issues. 1) I had to up the fcgid timing. This allowed me to get to step 15 rather than being stuck on step 5.

rickchristie.com/blog/2011/note/apache-mod_fcgid-read-data-timeout-error/

2) I had to up the php memory_limit

The problem with this second issue is that I upped the memory_limit to 256M then to 512M but it's still exhausting the memory and crashing. This sounds like a memory leak with this script. How much memory I should allocate for this script to run all the way through?

###error.log###
[Tue Jan 08 11:13:28 2013] [warn] [client 172.16.15.2] mod_fcgid: stderr: PHP Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 8079581 bytes) in /var/www/libraries/joomla/filesystem/file.php on line 256, referer: 172.16.15.21/administrator/index.php?option=com_fleximigrate
Last Edit: 08 Jan 2013 20:38 by warnerp.
netassopro NetAssoPro OFFLINE
Post : 615

Re: stuck on backup

#1111 09 Jan 2013 09:53
Thx for the information.
I imagine that you have a huge database to have this problem of memory.
To perform a better script in flexi migrate, i have to truncate the script or manage the memory into the code, but i will need time for that, and i have never enough time in this moment...
So easy solution that i saw (not very good but no choice) it's to up temporally your memory limit (like 4 go to be sure ) during the migration.
warnerp warnerp OFFLINE
Post : 12

Re: stuck on backup

#1112 09 Jan 2013 19:14
Our entire database is only 55MB. The Flexicontent tables are only 33MB. What exactly is this script capturing/doing that would need so much memory?
Last Edit: 09 Jan 2013 19:20 by warnerp.
warnerp warnerp OFFLINE
Post : 12

Re: stuck on backup

#1113 09 Jan 2013 23:48
FINALLY able to get it to complete. I had to completely disable the php
memory_limit (-1)

I upped my test Virtual Machines memory to 4GB
I also set
FcgidIOTimeout 7200
FcgidBusyTimeout 3600

Final zip file size was 886MB
warnerp warnerp OFFLINE
Post : 12

Re: stuck on backup

#1114 10 Jan 2013 01:39
i just ran this a second time with these configurations. I watched it with top and one php-cgi instance ran for 14 min and peaked at 97-98% memory.
I just looked at your helper.php . It looks like in step 15 you are reading all the files into one array then zipping the entire thing in one step. What if you tried creating the zip file, then create the array, then in the loop, loop through the list and append each file to the zip file then close it. I'm just a novice php developer but I might try to rewrite this to see if it's any more memory friendly. I'd imagine it might take a bit longer to run.
netassopro NetAssoPro OFFLINE
Post : 615

Re: stuck on backup

#1116 10 Jan 2013 09:43
Thank's to study a little my code. I make this component a little quickly to help some users. It's sure that we can do a better code to improve performance. If you have a little time and you want to improve it, it will help everybody. And i will add your update without problem (and of course i will add you in co-author).
warnerp warnerp OFFLINE
Post : 12

Re: stuck on backup

#1118 10 Jan 2013 23:18
Here's what I was able to come up with. I set my VM to 2GB with the PHP memory_limit to 256MB (left the fcgi settings as mentioned earlier). the PHP FCGI process only hits about 2% memory. It also takes a whole lot less time to complete. I haven't tested this with the import piece yet but from what I can tell, it's backing up identically as your's does. I'm sure there's some room to improve it, but it is doing what I need within the constraints that I need it too.

That last if statement should probably have your log/error messages set rather than just return false.

###helper.php
function step15(&$log)
{
$folder_path = JPATH_SITE . DS . 'images' . DS . 'fleximigrate';
$new_folder_name_final = $folder_path . DS . 'fleximigrate.zip';

ini_set("max_execution_time", 300);

$zip = new ZipArchive();

if ($zip->open($new_folder_name_final, ZIPARCHIVE::CREATE) !== TRUE) {
return false;
}

$iterator = new RecursiveIteratorIterator(new RecursiveDirectoryIterator($folder_path, FilesystemIterator::SKIP_DOTS));

foreach ($iterator as $key) {
$key = str_replace('\\', '/', $key);
if (!is_dir($key))
{
if(!$zip->addFile($key, substr($key, strlen($folder_path) - strlen($key)))) return false;
}
}
$zip->close();
}
Last Edit: 10 Jan 2013 23:22 by warnerp.
netassopro NetAssoPro OFFLINE
Post : 615

Re: stuck on backup

#1119 11 Jan 2013 09:38
It seems good. I added this code in a new release (that you can download now).
And i offer you an access to all our extension to thanks you.

Latest comments

  • FLEXICalendar
    By Griffouilles on 21/05/13 13:08
    got same question XD
  • FLEXI Migrate
    By NetAssoPro on 27/03/13 11:58
    Merci de reporter vos problèmes techniques dans l'espace dédié : le forum de support.
  • FLEXI Migrate
    By Xavier Lemoine on 26/03/13 17:46
    Impossible d'obtenir une migration fonctionnelle : après migration, Flexicontent reste bloqué sur "I...

Latest Posts

  • Import : Modification d'article
    By Acadprof on 22/05/2013 09:57
    Bonjour, Voila mon probléme : Mes imports se déroule bien avec mon XML qui crée les articles automatiquement sans aucun soucis. Par contre pour
  • demande d'amélioration si possible
    By rottenberg on 22/05/2013 19:24
    Bonjour, J'ai rencontré des problèmes lors de la définition du champ de cle primaire. Pourrait-on dans la liste des champs sélectionné n'avoir que
  • problème date de debut de publication dans Fleximp
    By rottenberg on 22/05/2013 19:52
    Je viens d'effectuer un test en automatique avec le type XML. J'ai aussi le problème de date de début de promotion qui n'est pas pris en compte. D

Most Downloaded