SD 2.1: error processing extended attributes: Cannot allocate memory
 

Just upgraded to 2.1. When it ran, I recieved 5,000 syslog lines of....

" /Applications/SuperDuper!.app/Contents/MacOS/SDCopy: error processing extended attributes: Cannot allocate memory"

Nothing has changed in my configuration, except the upgrade to 2.1.

Ideas?

This is a (pointless) diagnostic printed by an OSX call we're making; it's quite annoying, but we need to make the call to properly copy certain types of metadata...

Thanks for the response. However, today this generated 56,000 lines in my syslog. Would it be possible to provide a link to download the previous 2.0.1 version, that did not induce these messages? Thanks.


EDIT: Found 2.0.1 at download.com

While you might have had a lot of logging during an erase-then-copy, subsequent Smart Updates will log far, far less... I'd really suggest sticking with 2.1...

I also found thousands of similar messages in the system log asl.log the first time running version 2.1 -- this was a smart update. The messages are like this:

[Sender /Applications/SuperDuper Folder/SuperDuper!.app/Contents/MacOS/SDCopy] [PID -1] [Message error processing extended attributes: Invalid argument] [Level 4] [UID -2] [GID -2] [Host xxxxxxxx]

xxxxxxx is the name of the host computer

This asl.log file is 18 Megabytes and it's mostly SD messages like this! Is there something wrong or is this expected? The size of the file worries me. I haven't tried to boot yet from this clone but does this look normal?

At present, there's nothing we can do right now, unfortunately, but we're looking again. The call that does the EA copying are logging in a weird way when they find unusual structures on the drive.

It might help to repair the startup volume -- not permissions, but the drive itself. You'll usually want to boot from your Tiger install DVD and run Disk Utility from the Installer or Utilities menu to do that.

(For the curious, those who have permission from Apple can look at where the logging is coming from by navigating to this page -- the errors are coming from copyfile_xattr, which is called from SD under Tiger, via copyfile, to copy the EAs.)

These have all been Smart Updates. The first one was 5,000 lines. The next 3 were all 56,000 lines.

OK. Try what I suggested above and repair the drive. Make sure to only use Tiger-qualified tools: repairing with old versions of DW (etc) might be corrupting the EA part of the directory structure.

I did the following .....

1) Booted from Tiger install CD and ran Disk Utility - Repair Disk. The results were: "No Repairs Necesary".

2) Booted normally and ran SD 2.1 Smart Update. Recieved 53,100 lines in the syslog.

3) Ran Smart Update again and this time it added 56,300 lines to the syslog.


So, no change. Actually, first glance is that it gets worse each time it runs.

I also ran Disk Utility (10.4.4 OS X version) from another drive I booted from and the drive was ok. I think many users are not typically checking their system log files' size, and the asl.log is hidden away in /private/var/log.

If the asl.log file grows by ~ 18 Megabytes every time SD is run, that might not work well for me. Does the asl.log file get "recycled" every so often from the daily maintenance jobs? I ran DAILY and it didn't touch the asl.log file, although maybe that's because it wasn't very old. What is the criterion for DAILY to shrink the size of this file?

Another potential workaround -- is there a way to manually prune this asl.log file (which is just lines of text of these messages), say every several days, if it continues to grow like this? I'd hesitate to delete it or anything in /private/var unless I knew it were safe, and I suspect it is open for write during normal operations so doing something to it manually might cause some conflict if not done properly.

We're continuing to look into this; I'll let you know if we find a workaround.

Unfortunately, I don't really know what is going on here, but have a thought. Are any of you having these problems running Macaroni? Dale Meyn.

I don't think this has to do with Macaroni, Dale; there's clearly some case that's confusing the EA copying in the call we're using, which is making it log. We just have to try to work around that problem in the call.

It is, though, just (excessive) logging.

I did some research on this log file, asl.log, and it turns out that some other programs can result in huge output to asl.log. For instance, Adobe Acrobat, see

http://www.24help.info/adobe-acrobat...e-6.html?pp=10

This link also includes an example of how the terminal command call to syslog can be used to shrink the size of the log file. However this requires using root privileges (sudo), but if one mistypes or makes mistakes when doing this, it could have unpleasant consequences. So it's probably not something that the typical user (like me) wants to do routinely.

The normal cron maintenance routines should eventually prune these log files, I have read, but when I look inside the "daily" script it looks like it does this only after 21 days, during which literally hundreds (or thousands) of megabytes might be piling up. I was getting 18 Megabytes in asl.log with each backup.

By the way, is this happening only on my computer, or have you or others confirmed that this asl.log file under Tiger (I was using 10.4.4) grows by as much as many megabytes with each execution of SuperDuper (my disk being backed up is ~ 20 Gigagbytes in size)? My OS has never been modified except through the Apple Software Updates, it's version 10.4.4 right now. Recall that the asl.log file is in a hidden directory, /private/var/log.

We've seen this same problem, and can reproduce it, but not 18MB per backup, no.

I'll see if I can write a little script that can be used to trim the log automatically for you after a SD! run -- that will at least help until we can work around the call that's doing the logging.

Stand by.

OK. Attached is a little "prune_log" script that should work.

To use, unzip the attachment to an appropriate folder. (I suggest your personal Applications folder, but it's up to you.) Ensure that it looks like an "Unix executable file" when viewed in the finder.

Now, open SuperDuper! In the Advanced tab of Options, there's a selection under "After copy" to run a shell script when the copy completes. Check the box, and select the "prune_log" executable in the open panel.

That'll automatically remove any warnings from your system log on completion, assuming there were no errors during the run. All of these entries in the log are warnings, not errors, generated by the call I've spoken of earlier in the thread. Logging to the regular SuperDuper!.log will continue as usual, and no pruning will be done if an error occurs.

(For those who are interested, I tried to limit the pruning to just the SDCopy entries, but while the regular expression matching worked fine to restrict viewing to those entries, it wouldn't work to prune them, and I couldn't hardwire the location of SDCopy, which did work, because the user could have SuperDuper! installed anywhere.

What I attempted was:

syslog -p -k Sender Req .\*SDCopy

Without the -p, that only shows SDCopy entries. With the -p, it should prune those entries, but doesn't, and gives no errors. You'll need to put "sudo " in front of that if you want to try in Terminal, at least with -p.)

I've tested this here and it does properly prune the asl.log of these entries; let me know if it helps you out.

Same problem! And my asl.log is 67 MB curently.

After the script, my asl.log is 359 Kb. But, I noticed that SuperDuper! 2.1 was using 430 Mb of memory while running.

wow .. Just looked at my asl.log and it's 92mb .. Ran the script (through Terminal) and pruned it to 700kb .. Thanks

430MB of memory doesn't mean much, actually, because it includes shared components and stuff... we're generally pretty frugal.

Thanks for the quick response with the script! That was really fast.

By the way, I don't think the memory usage noted above is serious nor is it connected to the writing of messages to the log file. The messages are probably written a line at a time, which should have no effect on memory.

Yes, Macaroni 2.0.7

FWIW ..... The 56,000 lines per Smart Update are in my /var/log/system.log. Since there is no "Level", the script has no impact.

/Applications/SuperDuper!.app/Contents/MacOS/SDCopy: error processing extended attributes: Cannot allocate memory

Thanks for continuing to look into this.

Yes, I understand. This is to deal with the asl.log, not the system.log. As I indicated, we're looking into working around the logging.

The message is exactly the same, though, and is a warning (which you can see with syslog).

Of course I am not sure but I feel like that the increasing memory problem seems like related with this excessive logging.

First, it started with SD! 2.1, and it was not the case with 2.0.1. If we think that SD copies files from one drive to another drive why the memory usage would increase continuously? Also, I am not sure whether the system writes the log messages one by one. There may be a buffer for those, again I am not sure.

May be this problem should be opened in another topic. But, as I noticed the memory usage is around 180 Mb after SD launched. And when SD starts copying it also starts and continue to increase. 180 ... 200 ... 220 ... 300 and upto 480 Mb. I wonder what will it be if the copy would continue. And it returns to 180-200 after SD finished with copying. 480-180= 300 Mb. If the author of SD would say that 300 Mb memory usage is normal for a copy operation, I would say nothing.

priller writes, "FWIW ..... The 56,000 lines per Smart Update are in my /var/log/system.log....."

Thou not necessarily the final solution, does not running an app like Macaroni (as you mention) flush/keep the size of your system.log file under control?

Marcaroni insures that the system.log file is rotated every 24 hours, it does not reduce the size of the daily's.

There were two issues discussed in this thread:

* asl.log growing in size due to many messages being written (harmless messages generated by Mac OS X Tiger)

* system.log growing due to similar messages

The asl.log issue is resolved by the neat little script (prune log) that was provided in an earlier post by the forum administrator. It works nicely.

I think the system.log file size issue is resolved by the DAILY maintenance script. I ran it manually and before doing so my system.log had been ~ 10 Meg having grown to that size just after running SD, but after the maintenance scripts the largest system.log (including the archived ones) was under 100 kbytes, which is nothing.

So SD has provided an automated cure to the asl.log growth (just add the script to your SD run), and it looks like to me that the automated Tiger maintenance scripts (which can be activated manually via terminal or via a free program like MacJanitor), which run every night automatically, somehow shrink the size of the system.log.

SD is not the only program that affects these log files this way. I think it's a defect in the way Mac OS X handles these log files. Until Apple cleans this up, I think it's really a non issue because Dave's prune-log script and the maintenance system scripts together remove the excess file sizes.

Log file rotation is not a resolution, it just masks the problem.

The size, in bytes, of the file is not the issue. It's the number of log entries. If you inject 56,000 lines of crap into the syslog, it makes it that much more difficult to look for real issues, like a hacker attack.

We're not satisfied with the prune_log workaround, and continue to try to figure out a way to stop it from logging entirely. Sorry for the time it's taking...

sdsl, I'm not sure I have any of the "asl.log" problems, and use macaroni so the system.log file is probably under control. I've tried looking for these files, can't find them, even using the "find" option in the "File" menu. Where are they, I'd like to look at them and size them up? The sum of all the "log" folders I can find (in Mac HD/library and Users/home/library) is about 1.5 MB, but these don't include "asi.log" nor "system.log". Dale Meyn.

Log files live in /var/log

That's my problem, where is /var/log? Can't find it even using "Find". Dale.

Don't use Find, Dale. Use Finder's "Go To Folder..." command, and enter "/private/var/log" as the path. (/var is a link)

Aaaaaah, so that's it! I haven't gotten used to the "Go to folder" business, it's something new to me. Thanks. Dale.

This script (which works well) was posted in mid-March. It shrunk the asl.log which was growing due to harmless error mesaages. Since then there have been I think two new versions of SuperDuper and also Mac OS has moved up to 10.4.7. Did either of these new versions eliminate the need for the script?

Yes, the script hasn't been necessary since the version that came out after this thread.