You can subscribe to this list here.
| 2002 |
Jan
|
Feb
|
Mar
|
Apr
(23) |
May
(45) |
Jun
(22) |
Jul
(11) |
Aug
(14) |
Sep
(38) |
Oct
(62) |
Nov
(34) |
Dec
(25) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2003 |
Jan
(88) |
Feb
(126) |
Mar
(182) |
Apr
(168) |
May
(244) |
Jun
(249) |
Jul
(206) |
Aug
(309) |
Sep
(250) |
Oct
(487) |
Nov
(620) |
Dec
(470) |
| 2004 |
Jan
(623) |
Feb
(671) |
Mar
(522) |
Apr
(841) |
May
(671) |
Jun
(638) |
Jul
(622) |
Aug
(544) |
Sep
(580) |
Oct
(621) |
Nov
(668) |
Dec
(575) |
| 2005 |
Jan
(440) |
Feb
(547) |
Mar
(626) |
Apr
(594) |
May
(597) |
Jun
(796) |
Jul
(760) |
Aug
(908) |
Sep
(948) |
Oct
(923) |
Nov
(1114) |
Dec
(686) |
| 2006 |
Jan
(897) |
Feb
(943) |
Mar
(1049) |
Apr
(882) |
May
(774) |
Jun
(736) |
Jul
(816) |
Aug
(798) |
Sep
(1000) |
Oct
(742) |
Nov
(881) |
Dec
(705) |
| 2007 |
Jan
(1086) |
Feb
(1085) |
Mar
(1132) |
Apr
(1011) |
May
(778) |
Jun
(847) |
Jul
(857) |
Aug
(865) |
Sep
(830) |
Oct
(939) |
Nov
(856) |
Dec
(543) |
| 2008 |
Jan
(933) |
Feb
(832) |
Mar
(772) |
Apr
(587) |
May
(723) |
Jun
(872) |
Jul
(962) |
Aug
(915) |
Sep
(766) |
Oct
(658) |
Nov
(780) |
Dec
(554) |
| 2009 |
Jan
(604) |
Feb
(766) |
Mar
(719) |
Apr
(745) |
May
(547) |
Jun
(554) |
Jul
(474) |
Aug
(338) |
Sep
(424) |
Oct
(670) |
Nov
(421) |
Dec
(510) |
| 2010 |
Jan
(732) |
Feb
(702) |
Mar
(693) |
Apr
(666) |
May
(556) |
Jun
(515) |
Jul
(553) |
Aug
(549) |
Sep
(344) |
Oct
(431) |
Nov
(437) |
Dec
(329) |
| 2011 |
Jan
(822) |
Feb
(540) |
Mar
(435) |
Apr
(437) |
May
(624) |
Jun
(458) |
Jul
(416) |
Aug
(395) |
Sep
(333) |
Oct
(280) |
Nov
(246) |
Dec
(324) |
| 2012 |
Jan
(340) |
Feb
(273) |
Mar
(429) |
Apr
(321) |
May
(311) |
Jun
(329) |
Jul
(201) |
Aug
(307) |
Sep
(263) |
Oct
(308) |
Nov
(315) |
Dec
(294) |
| 2013 |
Jan
(481) |
Feb
(337) |
Mar
(310) |
Apr
(269) |
May
(274) |
Jun
(231) |
Jul
(182) |
Aug
(214) |
Sep
(276) |
Oct
(178) |
Nov
(222) |
Dec
(150) |
| 2014 |
Jan
(135) |
Feb
(144) |
Mar
(218) |
Apr
(152) |
May
(312) |
Jun
(187) |
Jul
(197) |
Aug
(218) |
Sep
(241) |
Oct
(282) |
Nov
(292) |
Dec
(229) |
| 2015 |
Jan
(200) |
Feb
(133) |
Mar
(154) |
Apr
(162) |
May
(268) |
Jun
(274) |
Jul
(166) |
Aug
(311) |
Sep
(182) |
Oct
(236) |
Nov
(160) |
Dec
(216) |
| 2016 |
Jan
(187) |
Feb
(248) |
Mar
(259) |
Apr
(112) |
May
(203) |
Jun
(104) |
Jul
(156) |
Aug
(131) |
Sep
(135) |
Oct
(161) |
Nov
(179) |
Dec
(110) |
| 2017 |
Jan
(148) |
Feb
(96) |
Mar
(236) |
Apr
(99) |
May
(118) |
Jun
(156) |
Jul
(157) |
Aug
(204) |
Sep
(151) |
Oct
(152) |
Nov
(125) |
Dec
(58) |
| 2018 |
Jan
(127) |
Feb
(151) |
Mar
(119) |
Apr
(131) |
May
(170) |
Jun
(125) |
Jul
(103) |
Aug
(119) |
Sep
(143) |
Oct
(116) |
Nov
(141) |
Dec
(90) |
| 2019 |
Jan
(179) |
Feb
(126) |
Mar
(97) |
Apr
(135) |
May
(135) |
Jun
(110) |
Jul
(121) |
Aug
(61) |
Sep
(96) |
Oct
(48) |
Nov
(58) |
Dec
(105) |
| 2020 |
Jan
(116) |
Feb
(97) |
Mar
(114) |
Apr
(96) |
May
(154) |
Jun
(116) |
Jul
(76) |
Aug
(20) |
Sep
(68) |
Oct
(105) |
Nov
(33) |
Dec
(118) |
| 2021 |
Jan
(34) |
Feb
(81) |
Mar
(94) |
Apr
(74) |
May
(133) |
Jun
(86) |
Jul
(65) |
Aug
(44) |
Sep
(68) |
Oct
(56) |
Nov
(113) |
Dec
(195) |
| 2022 |
Jan
(135) |
Feb
(65) |
Mar
(108) |
Apr
(48) |
May
(102) |
Jun
(153) |
Jul
(89) |
Aug
(90) |
Sep
(135) |
Oct
(77) |
Nov
(85) |
Dec
(61) |
| 2023 |
Jan
(102) |
Feb
(62) |
Mar
(81) |
Apr
(103) |
May
(71) |
Jun
(45) |
Jul
(57) |
Aug
(60) |
Sep
(94) |
Oct
(104) |
Nov
(96) |
Dec
(68) |
| 2024 |
Jan
(107) |
Feb
(92) |
Mar
(91) |
Apr
(155) |
May
(78) |
Jun
(121) |
Jul
(64) |
Aug
(136) |
Sep
(108) |
Oct
(105) |
Nov
(124) |
Dec
(88) |
| 2025 |
Jan
(115) |
Feb
(95) |
Mar
(84) |
Apr
(23) |
May
(59) |
Jun
(89) |
Jul
(71) |
Aug
(59) |
Sep
(60) |
Oct
(24) |
Nov
(56) |
Dec
(58) |
| 2026 |
Jan
(62) |
Feb
(52) |
Mar
(44) |
Apr
(75) |
May
(30) |
Jun
(38) |
Jul
(67) |
Aug
(21) |
Sep
(31) |
Oct
(29) |
Nov
(64) |
Dec
|
|
From: Stefan G. W. <li...@xu...> - 2026-11-14 06:26:44
|
Am 03.11.26 um 11:15 schrieb Stefan G. Weichinger via Bacula-users: > Am 02.11.26 um 08:37 schrieb Eric Bollengier via Bacula-users: >> Hello Stefan, >> >> We have to update the project key for new operating systems and the >> instructions >> inside the repo should be updated to use the new syntax. Any estimates when that will happen? Am I understanding correctly that for being able to write to S3 storage I need the specific cloud-driver-package from the Bacula CE repos (and this package is not available from the Debian repos)? thank you, have a nice weekend |
|
From: Robert W. <rw...@ro...> - 2026-11-13 17:45:55
|
Awesome, thank you. ________________________________ From: Bill Arlofski Sent: Friday, November 13, 2026 12:09 PM To: Robert Webb; bac...@li... Subject: Re: [Bacula-users] LTO9 Drive Recommended Block Sizes On 11/13/26 6:53 AM, Robert Webb wrote: > Morning Bill, > > Well, it is morning here in my TZ. Looked to run your watch_bacula_SD-FD script and ran into a snag. It looks like your > script is hard coded for the binary and other files to be in /opt. My deployment on Debian has the bconsole and other > binaries in /usr/sbin. > > What do you suggest for getting this to work? Do I need to go in and modify the script for the Debian deployment directories? Hello Robert, Those are just defaults. :) If you run the script with -h, or no parameters at all, you can see all the command line options. You will want to use the -b (path to bconsole binary) and -c (path to bconsole config file) Hope this helps, Bill -- Bill Arlofski wa...@pr... |
|
From: Robert W. <rw...@ro...> - 2026-11-13 17:30:11
|
Morning Bill, Well, it is morning here in my TZ. Looked to run your watch_bacula_SD-FD script and ran into a snag. It looks like your script is hard coded for the binary and other files to be in /opt. My deployment on Debian has the bconsole and other binaries in /usr/sbin. What do you suggest for getting this to work? Do I need to go in and modify the script for the Debian deployment directories? Bacula directories: find / -name bacula /var/log/bacula /var/lib/bacula /var/bacula /usr/share/logwatch/scripts/services/bacula /usr/libexec/bacula /usr/lib/bacula /etc/bacula /run/bacula Robert ________________________________ From: Bill Arlofski Sent: Thursday, November 12, 2026 2:46 PM To: Robert Webb; bac...@li... Subject: Re: [Bacula-users] LTO9 Drive Recommended Block Sizes On 11/10/26 7:37 PM, Robert Webb wrote: > Hi Bill, > > Thanks for the pointer to your scripts. However, not sure which ones might be applicable as I am using a single drive setup > and no changer. Ahhh OK. Then I think only the bacula-tapealert, and not the mtx-changer-python one - unless and until you get a library. :) > Also, I never received your first email and it took hours for mine to actually come through. Yeah, strange. And for me, it looks like multiple copies of my emails yesterday were sent. 🤷 😀 BTW, the `watch_bacula_SD-FD` script might also be useful at times. I always have it running in a tmux pane alongside a bconsole session, and a couple other panes tailing some interesting (to me :) logs. Best regards, Bill -- Bill Arlofski wa...@pr... |
|
From: Bill A. <wa...@pr...> - 2026-11-13 17:09:30
|
On 11/13/26 6:53 AM, Robert Webb wrote: > Morning Bill, > > Well, it is morning here in my TZ. Looked to run your watch_bacula_SD-FD script and ran into a snag. It looks like your > script is hard coded for the binary and other files to be in /opt. My deployment on Debian has the bconsole and other > binaries in /usr/sbin. > > What do you suggest for getting this to work? Do I need to go in and modify the script for the Debian deployment directories? Hello Robert, Those are just defaults. :) If you run the script with -h, or no parameters at all, you can see all the command line options. You will want to use the -b (path to bconsole binary) and -c (path to bconsole config file) Hope this helps, Bill -- Bill Arlofski wa...@pr... |
|
From: Arno L. <al...@it...> - 2026-11-12 21:29:13
|
Hi Ken, Am 12.11.2026 um 21:46 schrieb kjohnson--- via Bacula-users: > Our backups are configured so that each run uses its own cartridge; > tomorrow's backup is never appended to today's. We have daily, monthly, and > off-site pools. > > A numbers of times I have seen the backup job stall on or before the first > write to the mounted tape. It doesn't fail, but it does not progress. > Twice now I have been able to resolve this situation by deleting that volume > from the pool, writing an EOF to the start of the tape, and then relabeling > the tape with the same name in the same pool. Starting the backup again on > that same tape works exactly as it should. > > Fuji tapes, HP drive. > > Does this set of symptoms suggest anything that might help me figure this > out and make it stop? Nothing obvious. What I propose is the following: First, describe which exact version of Bacula you run -- not just the version number, but also where it comes from. Linux distros have different build settings thatn what Bacula Systems provides, and if you build from source yourself, nobody except you can know exactly what you run. Observe and record the status of Director, File Daemon and Storage Daemon during those situations. Then, you can tell the SD to trace all it does using a command such as setdebug storage=<the-storage-name> level=400 trace=1 options=t which should then result in a trace file in the SD's working directory including time stamps. Level 400 is going to result in a lot of information. You should familiarize yourself with this, and then try to arrange capturing this sort of trace around the time jobs start. reading those lines may already provide clues, and sharing them here might lead to someone else developing an idea. If that doesn't lead anywhere, there's the option to run the Storage Daemon with strace attached when tracing it, which will create very detailed information but needs patience and experience to understand. Better start with the simple steps -- it's not unlikely you'll get a recommendation to upgrade and that might already fix things :-) Cheers, Arno > Thanks, > > Ken > > > > > _______________________________________________ > Bacula-users mailing list > Bac...@li... > https://lists.sourceforge.net/lists/listinfo/bacula-users -- Arno Lehmann IT-Service Lehmann Sandstr. 6, 49080 Osnabrück |
|
From: Bill A. <wa...@pr...> - 2026-11-12 21:24:10
|
On 11/12/26 1:46 PM, kjohnson--- via Bacula-users wrote: > > Our backups are configured so that each run uses its own cartridge; > tomorrow's backup is never appended to today's. We have daily, monthly, and > off-site pools. > > A numbers of times I have seen the backup job stall on or before the first > write to the mounted tape. It doesn't fail, but it does not progress. > Twice now I have been able to resolve this situation by deleting that volume > from the pool, writing an EOF to the start of the tape, and then relabeling > the tape with the same name in the same pool. Starting the backup again on > that same tape works exactly as it should. > > Fuji tapes, HP drive. > > Does this set of symptoms suggest anything that might help me figure this > out and make it stop? Hello Ken, Do you have a `status storage=xxx` output from a time when this situation occurs? If you don't, then we will have to wait until the next time this happens. At that time, p erhaps we can set the SD in to debug mode and gather a trace for a few minutes to see exactly what the SD is doing at that time. Best regards, Bill -- Bill Arlofski wa...@pr... |
|
From: <kjo...@ec...> - 2026-11-12 21:04:52
|
Our backups are configured so that each run uses its own cartridge; tomorrow's backup is never appended to today's. We have daily, monthly, and off-site pools. A numbers of times I have seen the backup job stall on or before the first write to the mounted tape. It doesn't fail, but it does not progress. Twice now I have been able to resolve this situation by deleting that volume from the pool, writing an EOF to the start of the tape, and then relabeling the tape with the same name in the same pool. Starting the backup again on that same tape works exactly as it should. Fuji tapes, HP drive. Does this set of symptoms suggest anything that might help me figure this out and make it stop? Thanks, Ken |
|
From: Bill A. <wa...@pr...> - 2026-11-12 19:46:20
|
On 11/10/26 7:37 PM, Robert Webb wrote: > Hi Bill, > > Thanks for the pointer to your scripts. However, not sure which ones might be applicable as I am using a single drive setup > and no changer. Ahhh OK. Then I think only the bacula-tapealert, and not the mtx-changer-python one - unless and until you get a library. :) > Also, I never received your first email and it took hours for mine to actually come through. Yeah, strange. And for me, it looks like multiple copies of my emails yesterday were sent. 🤷 😀 BTW, the `watch_bacula_SD-FD` script might also be useful at times. I always have it running in a tmux pane alongside a bconsole session, and a couple other panes tailing some interesting (to me :) logs. Best regards, Bill -- Bill Arlofski wa...@pr... |
|
From: Bill A. <wa...@pr...> - 2026-11-12 19:44:35
|
On 11/10/26 9:13 PM, Rob Gerber wrote: > Robert, > > Bill attached his tape test script to his first email. Perhaps your email service blocked the email because it contained a > script. I have renamed the script to not end in .sh, packed it in a password protected zip (password: bacula), and have re- > attached it here. Please let me know if you receive it. Thanks for helping out Robert(Gerber)! :) > Regarding the tapealert script, it provides essential information about errors reported by the tape drive. Things like "need > cleaning' or 'tape broken!'. Because you do not have a changer (which can monitor and report some of those things), I think > you especially could benefit from this upgraded script. Yeah, the Bacula SD has some nice, somewhat hidden, features built around the original `tapealert` script that has been shipping with Bacula since the beginning. When an `AlertCommand` and a `ControlDevice` are configured in a Bacula SD Device's configuration, the SD calls this script at the end of an operation and I believe also at the end of a job. The script prints out any drive alert codes to stdout when it is called, then, the SD logs these codes and some text about what each means to the job log - and depending on the type and criticality of the code(s), the SD can disable a drive device, disable a volume, and more. For example, when a drive requires cleaning *immediately* as in the snippet below, both the device and the volume that was in it at the time were disabled, and the SD failed the job (that should get someone's attention! :) ----8<---- 27-Oct 10:05 ol9-sd JobId 240228: Warning: Disabled Device "waa_a-Autochanger_Dev0" (/dev/tape/by-id/scsi-350223344ab001700-nst) due to tape alert=20. 27-Oct 10:05 ol9-sd JobId 240228: Warning: Disabled Volume "G05001TA" due to tape alert=20. 27-Oct 10:05 ol9-sd JobId 240228: Fatal error: Alert: Volume="G05001TA" alert=20: ERR=The tape drive needs cleaning: The drive thinks it has a head clog, or needs cleaning. 1. If the operation has stopped, eject the tape and clean the drive. 2. If the operation has not stopped, wait for it to finish and then clean the drive. Check the tape drive users manual for device specific cleaning instructions. ----8<---- My replacement tapealert script performs the exact same functions as the original, but because it can auto-detect the /dev/sg# device you shouldn't run into a situation where your tapealert checks are no longer working because the drive's SG node changed after a reboot - and you don't know they are not working. Also, it does some nice logging, and can even send an email when tape alerts are found. Plus, it could use some real-world testing. :) Best regards, Bill -- Bill Arlofski wa...@pr... |
|
From: Marcin H. <gan...@gm...> - 2026-11-12 17:43:51
|
Hello everyone, I just wanted to let you know that we've released Bacularis 7.1.0. This release introduces two new plugins: one for InfluxDB database backups and another for Malware & Pattern Detection. If you're interested, I'd like to invite you to take a look at the 7.1.0 release notes. You can find them here: https://bacularis.app/news/189/36/New-release-Bacularis-7.1.0 We wish you smooth installations and upgrades! On behalf of the Bacularis Team, Marcin Haba |
|
From: Eric B. <er...@ba...> - 2026-11-12 07:16:39
|
Hello Josh, On 11/11/26 18:36, Josh Fisher via Bacula-users wrote: > > > On 11/10/26 10:20 AM, Lloyd Brown via Bacula-users wrote: >> The difficulty here is that, if we use one of the tools to keep no more than >> one job per disk volume (eg. `Maximum Volume Jobs=1` or `Volume Use >> Duration=N` where N is less than the normal job cycle), we end up with smaller >> disk files, and more of them. In this or similar circumstances, how do people >> control their disk pools to prevent them from filling up? > > I don't think that is necessary. The important thing is that there not be > multiple volumes that the incremental job can write to at any single point in > time. For example, if there is a pool of available volumes for JOB_1 and > multiple drives on which they can be loaded, such as when using a virtual disk > autochanger, then JOB_1 may select VOLUME_1 and Monday, then on Tuesday other > simultaneously running jobs may force JOB_1 to select Volume_2 on Tuesday, then > go back to writing on VOLUME_1 and Wednesday. The incremental jobs will run > fine, but a virtual full will not be possible. Thanks for the example, it's a little bit more complicated, but the spirit is here. The first job is on Volume1, if the second job starts on Volume2, fill it up, and switch back to Volume1 to finish, we end up with a cycle, one file might be spread between the two volumes, and this is causing the cycle. This is a rare case and restore jobs will handle the situation automatically. The BSR handling part is working pretty well and is one of the most critical piece of Bacula, I have ideas to solve this issue but no time for the moment. The configuration can prevent this problem, and if it's important, it is possible to migrate the jobs that are creating the loop to an other set of volumes to fix the situation and restart the VF. > One solution is to assign each job its own pool for incremental jobs. This > ensures that the job keeps writing to the same volume as long as it is active, > then never reuses it until after it is recycled. > > Another way is to limit volume use to 1 day (assuming you don't run incrementals > more often), which also prevents a previously used volume from being used again > after another volume is used, but in a different way. > Thanks to use Bacula, Best Regards, Eric |
|
From: Robert W. <rw...@ro...> - 2026-11-11 23:19:15
|
Sorry for the second email. The one sent this morning with the delay was from the web version of outlook. This one and the last one is from the Outlook app on my computer. Robert ________________________________ From: Rob Gerber <ro...@cr...> Sent: Wednesday, November 11, 2026 3:27 PM To: Robert Webb <rw...@ro...> Cc: Bill Arlofski <wa...@pr...>; bac...@li... <bac...@li...> Subject: Re: [Bacula-users] LTO9 Drive Recommended Block Sizes Hello Robert, Bacularis is quite handy. :) When you have an opportunity, you may find Bill's bacula backup report script useful as well. When I implemented it (approximately 1-1.5 years ago), I did find that it doesn't handle SSL communication, so you'd probably need to set up postfix or a similar local mail service, then point the script at that infrastructure. I reviewed the headers for your first email. You sent it around the time you stated, it was received by one of your mail carrier's servers, then held for 4 hours before being dispatched and rapidly relayed to the list. Not sure how you access headers in your email client, but once you get the full message do a ctrl+f for ' Nov' and/or 'Received:' Received: from SJ5PPFF2528D033.namprd15.prod.outlook.com<http://SJ5PPFF2528D033.namprd15.prod.outlook.com> (2603:10b6:a0f:fc02::b17) by SJ0PR15MB4759.namprd15.prod.outlook.com<http://SJ0PR15MB4759.namprd15.prod.outlook.com> (2603:10b6:a03:37c::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.615.8; Tue, 10 Nov 2026 16:53:44 +0000 Received: from mail-northcentralusazon11022119.outbound.protection.outlook.com<http://mail-northcentralusazon11022119.outbound.protection.outlook.com> ([40.107.200.119] helo=CH5PR02CU005.outbound.protection.outlook.com<http://CH5PR02CU005.outbound.protection.outlook.com>) by sfi-mx-1.v28.lw.sourceforge.com<http://sfi-mx-1.v28.lw.sourceforge.com> with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.100.1) id 1xQvHC-000000004EP-38hW for bac...@li...<mailto:bac...@li...>; Tue, 10 Nov 2026 23:29:19 +0000 Your most recent email's headers insist that you sent it at 13:31 UTC, with a less than 1 minute transit time to the mailing list servers. Are you certain you send your most recent email at 12:21 UTC? I checked the headers for your email to the list back in April 2026, and I see a similar delay. The intial email was delayed transiting from the outlook outbound to the sourceforge server by about 3 hours. In your subsequent reply to bill (which for me, was to the list and not directly to my address), I saw a delay of 1.5 hours, again between the outbound outlook server to the sourceforge server. I would generally suspect that this is due to something on the outlook side, but I cannot be certain. For example there might be a mechanism whereby the sourceforge server refuses to accept a promptly submitted email because it is overloaded, but later accepts the same email when the sending server retries. I'm fairly confident something between you and the list ate Bill's first email, likely because of the script. If you would like to troubleshoot this further I am open to an off-list email exchange. In particular, I'd like to send you a few dummy script files just to see which emails you do and don't get. Feel free to email me directly offlist if you would like to do that. This email is being sent at 20:27 UTC (Wednesday Nov 11th). Regards, Robert Gerber 402-237-8692 ro...@cr...<mailto:ro...@cr...> On Wed, Nov 11, 2026 at 7:31 AM Robert Webb <rw...@ro...<mailto:rw...@ro...>> wrote: Rob, Thanks for the feedback and attaching the script. Once this big restore eventually finishes, I will certainly run Bill's tape test and in the tapealert script. I looked this morning in my spam filtering from yesterday and do not see the first email as being even received. Still trying to get up to speed with Bacula, but am using bacularis as my front end right now. My first email was sent at 16:53 UTC yesterday and I didn't get a delivered copy until 23:30 UTC. Received Bill's second email at 00:02 UTC. Sending this at 12:31 UTC November 11th. Robert ________________________________ From: Rob Gerber <ro...@cr...<mailto:ro...@cr...>> Sent: Tuesday, November 10, 2026 11:13 PM To: Robert Webb <rw...@ro...<mailto:rw...@ro...>> Cc: Bill Arlofski <wa...@pr...<mailto:wa...@pr...>>; bac...@li...<mailto:bac...@li...> <bac...@li...<mailto:bac...@li...>> Subject: Re: [Bacula-users] LTO9 Drive Recommended Block Sizes Robert, Bill attached his tape test script to his first email. Perhaps your email service blocked the email because it contained a script. I have renamed the script to not end in .sh, packed it in a password protected zip (password: bacula), and have re-attached it here. Please let me know if you receive it. Regarding the tapealert script, it provides essential information about errors reported by the tape drive. Things like "need cleaning' or 'tape broken!'. Because you do not have a changer (which can monitor and report some of those things), I think you especially could benefit from this upgraded script. I agree, mtx-changer probably isn't useful to you. When you reply via email, please state the time in UTC. I will let you know what the received time is on my end. A reply-all should take the mailing list out of the equation, since you'll also be directly replying to me. Maybe you are having email server difficulties. As you may already know, we can also check the mailing list website for delivery status of an email, though I'm not sure how quickly those emails are displayed. Incidentally, I received your first email from the list at 23:30 UTC. Regards, Robert Gerber 402-237-8692<tel:(402)%20237-8692> ro...@cr...<mailto:ro...@cr...> On Tue, Nov 10, 2026 at 8:52 PM Robert Webb <rw...@ro...<mailto:rw...@ro...>> wrote: Hi Bill, Thanks for the pointer to your scripts. However, not sure which ones might be applicable as I am using a single drive setup and no changer. Also, I never received your first email and it took hours for mine to actually come through. Robert Get Outlook for Android<https://aka.ms/AAb9ysg> ________________________________ From: Bill Arlofski via Bacula-users <bac...@li...<mailto:bac...@li...>> Sent: Tuesday, 10 November 2026 19:01:27 To: bac...@li...<mailto:bac...@li...> <bac...@li...<mailto:bac...@li...>> Subject: Re: [Bacula-users] LTO9 Drive Recommended Block Sizes Hello Robert, Sorry for the two rapid replies, but in addition to my last email, I thought I might point you to two "drop-in" replacement scripts I have written that have some better logging and additional features as compared to the originals shipped with Bacula for 20+ years. You might want to have a look at `mtx-changer-python.py` and `bacula-tapealert.py`, available in my Github account here: https://github.com/waa Hope this helps! I look forward to news about your tape drive + Bacula experiences! :) Bill -- Bill Arlofski wa...@pr...<mailto:wa...@pr...> _______________________________________________ Bacula-users mailing list Bac...@li...<mailto:Bac...@li...> https://lists.sourceforge.net/lists/listinfo/bacula-users |
|
From: Robert W. <rw...@ro...> - 2026-11-11 20:44:41
|
Rob, I will do some digging on my end. Using Outlook as my client tied into a Microsoft Exchange Online subscription. It is Microsoft hosted on my said, so maybe enough said at this point. 🙂 I will look though some of the scripts. Thanks for taking the time to look into the email headers. Sending 20:44 UTC Robert ________________________________ From: Rob Gerber <ro...@cr...> Sent: Wednesday, November 11, 2026 3:27 PM To: Robert Webb <rw...@ro...> Cc: Bill Arlofski <wa...@pr...>; bac...@li... <bac...@li...> Subject: Re: [Bacula-users] LTO9 Drive Recommended Block Sizes Hello Robert, Bacularis is quite handy. :) When you have an opportunity, you may find Bill's bacula backup report script useful as well. When I implemented it (approximately 1-1.5 years ago), I did find that it doesn't handle SSL communication, so you'd probably need to set up postfix or a similar local mail service, then point the script at that infrastructure. I reviewed the headers for your first email. You sent it around the time you stated, it was received by one of your mail carrier's servers, then held for 4 hours before being dispatched and rapidly relayed to the list. Not sure how you access headers in your email client, but once you get the full message do a ctrl+f for ' Nov' and/or 'Received:' Received: from SJ5PPFF2528D033.namprd15.prod.outlook.com<http://SJ5PPFF2528D033.namprd15.prod.outlook.com> (2603:10b6:a0f:fc02::b17) by SJ0PR15MB4759.namprd15.prod.outlook.com<http://SJ0PR15MB4759.namprd15.prod.outlook.com> (2603:10b6:a03:37c::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.615.8; Tue, 10 Nov 2026 16:53:44 +0000 Received: from mail-northcentralusazon11022119.outbound.protection.outlook.com<http://mail-northcentralusazon11022119.outbound.protection.outlook.com> ([40.107.200.119] helo=CH5PR02CU005.outbound.protection.outlook.com<http://CH5PR02CU005.outbound.protection.outlook.com>) by sfi-mx-1.v28.lw.sourceforge.com<http://sfi-mx-1.v28.lw.sourceforge.com> with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.100.1) id 1xQvHC-000000004EP-38hW for bac...@li...<mailto:bac...@li...>; Tue, 10 Nov 2026 23:29:19 +0000 Your most recent email's headers insist that you sent it at 13:31 UTC, with a less than 1 minute transit time to the mailing list servers. Are you certain you send your most recent email at 12:21 UTC? I checked the headers for your email to the list back in April 2026, and I see a similar delay. The intial email was delayed transiting from the outlook outbound to the sourceforge server by about 3 hours. In your subsequent reply to bill (which for me, was to the list and not directly to my address), I saw a delay of 1.5 hours, again between the outbound outlook server to the sourceforge server. I would generally suspect that this is due to something on the outlook side, but I cannot be certain. For example there might be a mechanism whereby the sourceforge server refuses to accept a promptly submitted email because it is overloaded, but later accepts the same email when the sending server retries. I'm fairly confident something between you and the list ate Bill's first email, likely because of the script. If you would like to troubleshoot this further I am open to an off-list email exchange. In particular, I'd like to send you a few dummy script files just to see which emails you do and don't get. Feel free to email me directly offlist if you would like to do that. This email is being sent at 20:27 UTC (Wednesday Nov 11th). Regards, Robert Gerber 402-237-8692 ro...@cr...<mailto:ro...@cr...> On Wed, Nov 11, 2026 at 7:31 AM Robert Webb <rw...@ro...<mailto:rw...@ro...>> wrote: Rob, Thanks for the feedback and attaching the script. Once this big restore eventually finishes, I will certainly run Bill's tape test and in the tapealert script. I looked this morning in my spam filtering from yesterday and do not see the first email as being even received. Still trying to get up to speed with Bacula, but am using bacularis as my front end right now. My first email was sent at 16:53 UTC yesterday and I didn't get a delivered copy until 23:30 UTC. Received Bill's second email at 00:02 UTC. Sending this at 12:31 UTC November 11th. Robert ________________________________ From: Rob Gerber <ro...@cr...<mailto:ro...@cr...>> Sent: Tuesday, November 10, 2026 11:13 PM To: Robert Webb <rw...@ro...<mailto:rw...@ro...>> Cc: Bill Arlofski <wa...@pr...<mailto:wa...@pr...>>; bac...@li...<mailto:bac...@li...> <bac...@li...<mailto:bac...@li...>> Subject: Re: [Bacula-users] LTO9 Drive Recommended Block Sizes Robert, Bill attached his tape test script to his first email. Perhaps your email service blocked the email because it contained a script. I have renamed the script to not end in .sh, packed it in a password protected zip (password: bacula), and have re-attached it here. Please let me know if you receive it. Regarding the tapealert script, it provides essential information about errors reported by the tape drive. Things like "need cleaning' or 'tape broken!'. Because you do not have a changer (which can monitor and report some of those things), I think you especially could benefit from this upgraded script. I agree, mtx-changer probably isn't useful to you. When you reply via email, please state the time in UTC. I will let you know what the received time is on my end. A reply-all should take the mailing list out of the equation, since you'll also be directly replying to me. Maybe you are having email server difficulties. As you may already know, we can also check the mailing list website for delivery status of an email, though I'm not sure how quickly those emails are displayed. Incidentally, I received your first email from the list at 23:30 UTC. Regards, Robert Gerber 402-237-8692<tel:(402)%20237-8692> ro...@cr...<mailto:ro...@cr...> On Tue, Nov 10, 2026 at 8:52 PM Robert Webb <rw...@ro...<mailto:rw...@ro...>> wrote: Hi Bill, Thanks for the pointer to your scripts. However, not sure which ones might be applicable as I am using a single drive setup and no changer. Also, I never received your first email and it took hours for mine to actually come through. Robert Get Outlook for Android<https://aka.ms/AAb9ysg> ________________________________ From: Bill Arlofski via Bacula-users <bac...@li...<mailto:bac...@li...>> Sent: Tuesday, 10 November 2026 19:01:27 To: bac...@li...<mailto:bac...@li...> <bac...@li...<mailto:bac...@li...>> Subject: Re: [Bacula-users] LTO9 Drive Recommended Block Sizes Hello Robert, Sorry for the two rapid replies, but in addition to my last email, I thought I might point you to two "drop-in" replacement scripts I have written that have some better logging and additional features as compared to the originals shipped with Bacula for 20+ years. You might want to have a look at `mtx-changer-python.py` and `bacula-tapealert.py`, available in my Github account here: https://github.com/waa Hope this helps! I look forward to news about your tape drive + Bacula experiences! :) Bill -- Bill Arlofski wa...@pr...<mailto:wa...@pr...> _______________________________________________ Bacula-users mailing list Bac...@li...<mailto:Bac...@li...> https://lists.sourceforge.net/lists/listinfo/bacula-users |
|
From: Rob G. <ro...@cr...> - 2026-11-11 20:28:00
|
Hello Robert, Bacularis is quite handy. :) When you have an opportunity, you may find Bill's bacula backup report script useful as well. When I implemented it (approximately 1-1.5 years ago), I did find that it doesn't handle SSL communication, so you'd probably need to set up postfix or a similar local mail service, then point the script at that infrastructure. I reviewed the headers for your first email. You sent it around the time you stated, it was received by one of your mail carrier's servers, then held for 4 hours before being dispatched and rapidly relayed to the list. Not sure how you access headers in your email client, but once you get the full message do a ctrl+f for ' Nov' and/or 'Received:' *Received*: from SJ5PPFF2528D033.namprd15.prod.outlook.com (2603:10b6:a0f:fc02::b17) *by SJ0PR15MB4759.namprd15.prod.outlook.com <http://SJ0PR15MB4759.namprd15.prod.outlook.com>* (2603:10b6:a03:37c::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.615.8; *Tue, 10 Nov 2026 16:53:44 +0000* *Received: from mail-northcentralusazon11022119.outbound.protection.outlook.com <http://mail-northcentralusazon11022119.outbound.protection.outlook.com>* ([40.107.200.119] helo=CH5PR02CU005.outbound.protection.outlook.com) *by sfi-mx-1.v28.lw.sourceforge.com <http://sfi-mx-1.v28.lw.sourceforge.com>* with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.100.1) id 1xQvHC-000000004EP-38hW for bac...@li...; *Tue, 10 Nov 2026 23:29:19 +0000* Your most recent email's headers insist that you sent it at 13:31 UTC, with a less than 1 minute transit time to the mailing list servers. Are you certain you send your most recent email at 12:21 UTC? I checked the headers for your email to the list back in April 2026, and I see a similar delay. The intial email was delayed transiting from the outlook outbound to the sourceforge server by about 3 hours. In your subsequent reply to bill (which for me, was to the list and not directly to my address), I saw a delay of 1.5 hours, again between the outbound outlook server to the sourceforge server. I would generally suspect that this is due to something on the outlook side, but I cannot be certain. For example there might be a mechanism whereby the sourceforge server refuses to accept a promptly submitted email because it is overloaded, but later accepts the same email when the sending server retries. I'm fairly confident something between you and the list ate Bill's first email, likely because of the script. If you would like to troubleshoot this further I am open to an off-list email exchange. In particular, I'd like to send you a few dummy script files just to see which emails you do and don't get. Feel free to email me directly offlist if you would like to do that. This email is being sent at 20:27 UTC (Wednesday Nov 11th). Regards, Robert Gerber 402-237-8692 ro...@cr... On Wed, Nov 11, 2026 at 7:31 AM Robert Webb <rw...@ro...> wrote: > Rob, > > Thanks for the feedback and attaching the script. Once this big restore > eventually finishes, I will certainly run Bill's tape test and in the > tapealert script. I looked this morning in my spam filtering from yesterday > and do not see the first email as being even received. > > Still trying to get up to speed with Bacula, but am using bacularis as my > front end right now. > > My first email was sent at 16:53 UTC yesterday and I didn't get a > delivered copy until 23:30 UTC. Received Bill's second email at 00:02 UTC. > > Sending this at 12:31 UTC November 11th. > > Robert > ------------------------------ > *From:* Rob Gerber <ro...@cr...> > *Sent:* Tuesday, November 10, 2026 11:13 PM > *To:* Robert Webb <rw...@ro...> > *Cc:* Bill Arlofski <wa...@pr...>; > bac...@li... <bac...@li...> > *Subject:* Re: [Bacula-users] LTO9 Drive Recommended Block Sizes > > Robert, > > Bill attached his tape test script to his first email. Perhaps your email > service blocked the email because it contained a script. I have renamed the > script to not end in .sh, packed it in a password protected zip (password: > bacula), and have re-attached it here. Please let me know if you receive it. > > Regarding the tapealert script, it provides essential information about > errors reported by the tape drive. Things like "need cleaning' or 'tape > broken!'. Because you do not have a changer (which can monitor and report > some of those things), I think you especially could benefit from this > upgraded script. > > I agree, mtx-changer probably isn't useful to you. > > When you reply via email, please state the time in UTC. I will let you > know what the received time is on my end. A reply-all should take the > mailing list out of the equation, since you'll also be directly replying to > me. Maybe you are having email server difficulties. As you may already > know, we can also check the mailing list website for delivery status of an > email, though I'm not sure how quickly those emails are displayed. > > Incidentally, I received your first email from the list at 23:30 UTC. > > Regards, > Robert Gerber > 402-237-8692 <(402)%20237-8692> > ro...@cr... > > > On Tue, Nov 10, 2026 at 8:52 PM Robert Webb <rw...@ro...> wrote: > > Hi Bill, > > Thanks for the pointer to your scripts. However, not sure which ones might > be applicable as I am using a single drive setup and no changer. > > Also, I never received your first email and it took hours for mine to > actually come through. > > Robert > > Get Outlook for Android <https://aka.ms/AAb9ysg> > ------------------------------ > *From:* Bill Arlofski via Bacula-users <bac...@li... > > > *Sent:* Tuesday, 10 November 2026 19:01:27 > *To:* bac...@li... < > bac...@li...> > *Subject:* Re: [Bacula-users] LTO9 Drive Recommended Block Sizes > > Hello Robert, > > Sorry for the two rapid replies, but in addition to my last email, I > thought I might point you to two "drop-in" replacement > scripts I have written that have some better logging and additional > features as compared to the originals shipped with Bacula > for 20+ years. > > You might want to have a look at `mtx-changer-python.py` and > `bacula-tapealert.py`, available in my Github account here: > > https://github.com/waa > > Hope this helps! > > I look forward to news about your tape drive + Bacula experiences! :) > > > Bill > > -- > Bill Arlofski > wa...@pr... > _______________________________________________ > Bacula-users mailing list > Bac...@li... > https://lists.sourceforge.net/lists/listinfo/bacula-users > > |
|
From: Josh F. <jf...@ja...> - 2026-11-11 17:52:53
|
On 11/10/26 10:20 AM, Lloyd Brown via Bacula-users wrote: > The difficulty here is that, if we use one of the tools to keep no > more than one job per disk volume (eg. `Maximum Volume Jobs=1` or > `Volume Use Duration=N` where N is less than the normal job cycle), we > end up with smaller disk files, and more of them. In this or similar > circumstances, how do people control their disk pools to prevent them > from filling up? I don't think that is necessary. The important thing is that there not be multiple volumes that the incremental job can write to at any single point in time. For example, if there is a pool of available volumes for JOB_1 and multiple drives on which they can be loaded, such as when using a virtual disk autochanger, then JOB_1 may select VOLUME_1 and Monday, then on Tuesday other simultaneously running jobs may force JOB_1 to select Volume_2 on Tuesday, then go back to writing on VOLUME_1 and Wednesday. The incremental jobs will run fine, but a virtual full will not be possible. One solution is to assign each job its own pool for incremental jobs. This ensures that the job keeps writing to the same volume as long as it is active, then never reuses it until after it is recycled. Another way is to limit volume use to 1 day (assuming you don't run incrementals more often), which also prevents a previously used volume from being used again after another volume is used, but in a different way. > > In the past (eg. Bacula 9.6.x, where VF worked great, BTW), I've used > a combination of `Maximum Volume Bytes` and `Maximum Volumes`. I > can't use the `Maximum Pool Bytes`, since that seems to only be > available for Bacula Enterprise. Therefore what? I just have to > guess the oversubscription ratio, and hope I'm right? > > > On 11/10/26 00:59, Eric Bollengier wrote: >> Hello Lloyd, >> >> On 11/9/26 18:12, Lloyd Brown via Bacula-users wrote: >>> Can anyone explain the reasoning behind the volume cycle detection >>> for VirtualFulls in Bacula 15.0.3? I'm running into problems like >>> what's shown below, and it's not clear to me why it's a problem. >>> >>> I understand that the VF code doesn't like to go back to a >>> previously-read volume, and that triggers this behavior. What I'm >>> struggling to understand is why it's a problem to revisit a volume. >> >> This is a limitation of Bacula, we read volume per volume >> (optimisation as Arno >> mentioned), and if a volume is used multiple times with other volumes >> in the >> middle, the virtual full is not possible. >> >> Ensure that your disk volume are used once per specific job, and >> everything >> will be fine. >> >> Hope it helps, >> Best Regards, >> Eric >>>> 07-Nov 17:05 backupdir-dir JobId 2671: Start Virtual Backup JobId >>>> 2671, Job=backupworker1job.2026-11-07_17.05.00_06 >>>> 07-Nov 17:05 backupdir-dir JobId 2671: Consolidating >>>> JobIds=2355,2378,2395,2424,2440,2476,2489,2507,2543,2566,2595,2614,2646,2660 >>>> 07-Nov 17:05 backupdir-dir JobId 2671: Found 88360 files to >>>> consolidate into Virtual Full. >>>> 07-Nov 17:05 backupdir-dir JobId 2671: Fatal error: Found a volume >>>> cycle in the bootstrap, Virtual Full is not possible on this Job >>>> 07-Nov 17:05 backupdir-dir JobId 2671: Error: Bacula backupdir-dir >>>> 15.0.3 (25Mar25): >>>> Build OS: x86_64-pc-linux-gnu redhat (Green >>>> JobId: 2671 >>>> Job: backupworker1job.2026-11-07_17.05.00_06 >>>> Backup Level: Virtual Full >>>> Client: "backupworker1-fd" 15.0.3 (25Mar25) >>>> x86_64-pc- linux-gnu,debian,trixie/sid >>>> FileSet: "backupworker-set" 2026-09-30 15:23:02 >>>> Pool: "Disk-granite2" (From Run NextPool >>>> override) >>>> Catalog: "MyCatalog" (From Client resource) >>>> Storage: "granite2-sd-disk" (From Run NextPool >>>> override) >>>> Scheduled time: 07-Nov-2026 17:05:00 >>>> Start time: 07-Nov-2026 17:00:04 >>>> End time: 07-Nov-2026 17:00:09 >>>> Elapsed time: 1 sec >>>> Priority: 10 >>>> SD Files Written: 0 >>>> SD Bytes Written: 0 (0 B) >>>> Rate: 0.0 KB/s >>>> Volume name(s): >>>> Volume Session Id: 0 >>>> Volume Session Time: 0 >>>> Last Volume Bytes: 0 (0 B) >>>> SD Errors: 0 >>>> SD termination status: >>>> Termination: *** Backup Error *** >>> >> |
|
From: Robert W. <rw...@ro...> - 2026-11-11 13:32:03
|
Rob, Thanks for the feedback and attaching the script. Once this big restore eventually finishes, I will certainly run Bill's tape test and in the tapealert script. I looked this morning in my spam filtering from yesterday and do not see the first email as being even received. Still trying to get up to speed with Bacula, but am using bacularis as my front end right now. My first email was sent at 16:53 UTC yesterday and I didn't get a delivered copy until 23:30 UTC. Received Bill's second email at 00:02 UTC. Sending this at 12:31 UTC November 11th. Robert ________________________________ From: Rob Gerber <ro...@cr...> Sent: Tuesday, November 10, 2026 11:13 PM To: Robert Webb <rw...@ro...> Cc: Bill Arlofski <wa...@pr...>; bac...@li... <bac...@li...> Subject: Re: [Bacula-users] LTO9 Drive Recommended Block Sizes Robert, Bill attached his tape test script to his first email. Perhaps your email service blocked the email because it contained a script. I have renamed the script to not end in .sh, packed it in a password protected zip (password: bacula), and have re-attached it here. Please let me know if you receive it. Regarding the tapealert script, it provides essential information about errors reported by the tape drive. Things like "need cleaning' or 'tape broken!'. Because you do not have a changer (which can monitor and report some of those things), I think you especially could benefit from this upgraded script. I agree, mtx-changer probably isn't useful to you. When you reply via email, please state the time in UTC. I will let you know what the received time is on my end. A reply-all should take the mailing list out of the equation, since you'll also be directly replying to me. Maybe you are having email server difficulties. As you may already know, we can also check the mailing list website for delivery status of an email, though I'm not sure how quickly those emails are displayed. Incidentally, I received your first email from the list at 23:30 UTC. Regards, Robert Gerber 402-237-8692 ro...@cr...<mailto:ro...@cr...> On Tue, Nov 10, 2026 at 8:52 PM Robert Webb <rw...@ro...<mailto:rw...@ro...>> wrote: Hi Bill, Thanks for the pointer to your scripts. However, not sure which ones might be applicable as I am using a single drive setup and no changer. Also, I never received your first email and it took hours for mine to actually come through. Robert Get Outlook for Android<https://aka.ms/AAb9ysg> ________________________________ From: Bill Arlofski via Bacula-users <bac...@li...<mailto:bac...@li...>> Sent: Tuesday, 10 November 2026 19:01:27 To: bac...@li...<mailto:bac...@li...> <bac...@li...<mailto:bac...@li...>> Subject: Re: [Bacula-users] LTO9 Drive Recommended Block Sizes Hello Robert, Sorry for the two rapid replies, but in addition to my last email, I thought I might point you to two "drop-in" replacement scripts I have written that have some better logging and additional features as compared to the originals shipped with Bacula for 20+ years. You might want to have a look at `mtx-changer-python.py` and `bacula-tapealert.py`, available in my Github account here: https://github.com/waa Hope this helps! I look forward to news about your tape drive + Bacula experiences! :) Bill -- Bill Arlofski wa...@pr...<mailto:wa...@pr...> _______________________________________________ Bacula-users mailing list Bac...@li...<mailto:Bac...@li...> https://lists.sourceforge.net/lists/listinfo/bacula-users |
|
From: Robert W. <rw...@ro...> - 2026-11-11 02:52:21
|
Hi Bill, Thanks for the pointer to your scripts. However, not sure which ones might be applicable as I am using a single drive setup and no changer. Also, I never received your first email and it took hours for mine to actually come through. Robert Get Outlook for Android<https://aka.ms/AAb9ysg> ________________________________ From: Bill Arlofski via Bacula-users <bac...@li...> Sent: Tuesday, 10 November 2026 19:01:27 To: bac...@li... <bac...@li...> Subject: Re: [Bacula-users] LTO9 Drive Recommended Block Sizes Hello Robert, Sorry for the two rapid replies, but in addition to my last email, I thought I might point you to two "drop-in" replacement scripts I have written that have some better logging and additional features as compared to the originals shipped with Bacula for 20+ years. You might want to have a look at `mtx-changer-python.py` and `bacula-tapealert.py`, available in my Github account here: https://github.com/waa Hope this helps! I look forward to news about your tape drive + Bacula experiences! :) Bill -- Bill Arlofski wa...@pr... |
|
From: Bill A. <wa...@pr...> - 2026-11-11 00:01:50
|
Hello Robert, Sorry for the two rapid replies, but in addition to my last email, I thought I might point you to two "drop-in" replacement scripts I have written that have some better logging and additional features as compared to the originals shipped with Bacula for 20+ years. You might want to have a look at `mtx-changer-python.py` and `bacula-tapealert.py`, available in my Github account here: https://github.com/waa Hope this helps! I look forward to news about your tape drive + Bacula experiences! :) Bill -- Bill Arlofski wa...@pr... |
|
From: Bill A. <wa...@pr...> - 2026-11-10 23:55:45
|
On 11/10/26 9:53 AM, Robert Webb wrote: > Not new to tape storage, but it has been many years since I have worked with them. > > Just set up a new Bacula deployment and ran a large backup of around 45TB's. Files are mostly large video files and then a > mix of of other files like photos. Noticed the backup settled out at around 114MB/s which I figured was good for what I was > backing up. > > Now trying to do a full restore and was perplexed as to why the restore was a bit slower. Went through a lot of diagnosis on > the Proxmox server setup with its ZFS storage, added a SLOG, and ultimately decided that storage was not the bottleneck, nor > was the 10G network connection. > > Through more reading, it seems that the default 64K block size being used to write to the LTO9 drive has a lot of overhead > and probably causing the slower backups and restores. So what is the better minimum and maximum black sizes? I have read 256k > to 512k is good for minimums and 1M to 2M for maximum. Looking for some confirmations before I make any changes and do > another full backup once all the restores are complete. > > > Thanks... Hello Robert, I have done a lot of testing in Quantum's lab to certify Bacula Enterprise with their LTO8/9/10 drives. A few years ago, after running the attached script, the best overall settings for the MaximumFileSize and MaximumBlockSize for SD Device settings were as follows: ----8<---- MaximumFileSize = 32GB MaximumBlockSize = 2097152 ----8<---- I let this script run over a weekend with more `file_sizes=` and `block_sizes=` variable settings than are in the attached script (edit as needed). In the end, a colleague of mine took the results and imported them into a spreadsheet which generated tables and graphs and those settings where the overall best for LTO8/9. The performance was just as good when I was testing LTO10 this summer. If you run this script and come up with different results (or the same ones :), I'd love to hear back from you! Note: MinimumBlockSize should never be set! Hope this helps, Bill -- Bill Arlofski wa...@pr... |
|
From: Robert W. <rw...@ro...> - 2026-11-10 23:29:28
|
Not new to tape storage, but it has been many years since I have worked with them. Just set up a new Bacula deployment and ran a large backup of around 45TB's. Files are mostly large video files and then a mix of of other files like photos. Noticed the backup settled out at around 114MB/s which I figured was good for what I was backing up. Now trying to do a full restore and was perplexed as to why the restore was a bit slower. Went through a lot of diagnosis on the Proxmox server setup with its ZFS storage, added a SLOG, and ultimately decided that storage was not the bottleneck, nor was the 10G network connection. Through more reading, it seems that the default 64K block size being used to write to the LTO9 drive has a lot of overhead and probably causing the slower backups and restores. So what is the better minimum and maximum black sizes? I have read 256k to 512k is good for minimums and 1M to 2M for maximum. Looking for some confirmations before I make any changes and do another full backup once all the restores are complete. Thanks... |
|
From: Arno L. <al...@it...> - 2026-11-10 20:11:39
|
Hi Lloyd, all, Am 10.11.2026 um 20:42 schrieb Lloyd Brown via Bacula-users: > On 11/10/26 12:28, Lloyd Brown via Bacula-users wrote: >> On 11/10/26 11:48, Arno Lehmann via Bacula-users wrote: >>>> This really gets to the crux of my original question. Why is the >>>> cycle bad? Suboptimal performance, perhaps, though I'd argue that >>>> for disks the difference is negligible. >>> >>> We stumbled over that because performance was so bad that users >>> actually couldn't complete their copy and virtual backups in decent >>> time, couldn't saturate their tape drives, and found that SD threads, >>> doing such jobs, wasted more than 90% (iirc) of a CPUs time going >>> through bootstrap records. Negligible only if you accepted a whole >>> university's backups to simply not take place any more 🙂 >> >> >> Interesting. This was occurring during the run of the VF? It's been a while, and I did not look up the actual case in our tickets, but given that it involved large jobs and involved writing to tape, I would assume it's been copy or migration jobs in that particular case. Unfortunately the users has since disabled their publicly visible monitoring system, otherwise I would have been able to point to very respectable performance today :-) >> We certainly didn't experience that. I wonder if the fact that the >> old 9.6.6 build had many more jobs per day, and had 100G disk volumes >> vs 500G currently on bacula 15, meant we avoided this effect without >> realizing it. That is quite likely -- if you just observe the size of the temporarily created bootstrap files, it may become obvious. In the case I'm thinking of, these were quite long due to exactly those topics you mentioned -- interleaved jobs and lots of volumes involved -- were surely the reason the problem became aparent, and the fact that tape drives are somewhat naturally limited in number and thus deserve proper monitoring of usage had a lot to do with the user noticing the issues. Anyway -- if the current situation, the available workarounds, and the fact that Bacula could just be better lead us to see improvements in that area, I'd be quite happy to qualify and verify any improvements; actually developing is most likely not something I should do :-) Cheers, Arno >> >> Lloyd >> > > This is additional speculation on my part, but I wonder also if it > depends on how interleaved the jobs' data are on the disk volumes. A > long time ago we had performance problems during a restore when it had > to do a lot of seeks within the disk volumes. We switched to having many > `Device` entries in the SD's config, each with `Maximum Concurrent Jobs > = 1`. The result is that interleaving within a specific job run almost > entirely disappeared. And that would mean many fewer transitions > between volumes when running the VF, and therefore fewer and likely > shorter cycles, if they occurred at all. > > Lloyd > > > > _______________________________________________ > Bacula-users mailing list > Bac...@li... > https://lists.sourceforge.net/lists/listinfo/bacula-users -- Arno Lehmann IT-Service Lehmann Sandstr. 6, 49080 Osnabrück |
|
From: Lloyd B. <llo...@by...> - 2026-11-10 19:42:47
|
On 11/10/26 12:28, Lloyd Brown via Bacula-users wrote: > On 11/10/26 11:48, Arno Lehmann via Bacula-users wrote: >>> This really gets to the crux of my original question. Why is the >>> cycle bad? Suboptimal performance, perhaps, though I'd argue that >>> for disks the difference is negligible. >> >> We stumbled over that because performance was so bad that users >> actually couldn't complete their copy and virtual backups in decent >> time, couldn't saturate their tape drives, and found that SD threads, >> doing such jobs, wasted more than 90% (iirc) of a CPUs time going >> through bootstrap records. Negligible only if you accepted a whole >> university's backups to simply not take place any more 🙂 > > > Interesting. This was occurring during the run of the VF? > > We certainly didn't experience that. I wonder if the fact that the > old 9.6.6 build had many more jobs per day, and had 100G disk volumes > vs 500G currently on bacula 15, meant we avoided this effect without > realizing it. > > > Lloyd > This is additional speculation on my part, but I wonder also if it depends on how interleaved the jobs' data are on the disk volumes. A long time ago we had performance problems during a restore when it had to do a lot of seeks within the disk volumes. We switched to having many `Device` entries in the SD's config, each with `Maximum Concurrent Jobs = 1`. The result is that interleaving within a specific job run almost entirely disappeared. And that would mean many fewer transitions between volumes when running the VF, and therefore fewer and likely shorter cycles, if they occurred at all. Lloyd |
|
From: Lloyd B. <llo...@by...> - 2026-11-10 19:29:13
|
On 11/10/26 11:48, Arno Lehmann via Bacula-users wrote: >> This really gets to the crux of my original question. Why is the >> cycle bad? Suboptimal performance, perhaps, though I'd argue that for >> disks the difference is negligible. > > We stumbled over that because performance was so bad that users > actually couldn't complete their copy and virtual backups in decent > time, couldn't saturate their tape drives, and found that SD threads, > doing such jobs, wasted more than 90% (iirc) of a CPUs time going > through bootstrap records. Negligible only if you accepted a whole > university's backups to simply not take place any more 🙂 Interesting. This was occurring during the run of the VF? We certainly didn't experience that. I wonder if the fact that the old 9.6.6 build had many more jobs per day, and had 100G disk volumes vs 500G currently on bacula 15, meant we avoided this effect without realizing it. Lloyd |
|
From: Martin S. <ma...@li...> - 2026-11-10 19:13:56
|
What are the columns of the file table? E.g. run "psql -U bacula bacula" and use the "\d file" command. If the columns include filenameid and not filename, then I suspect part of the update_postgres_tables script was never run, because that is supposed to fix these columns. __Martin >>>>> On Tue, 10 Nov 2026 11:50:00 -0500, Brian said: > > (I've been having some trouble with my outbound e-mail; please excuse if > this is a repost) > > Hello, > > This past weekend I upgraded a Debian 12 server to Debian 13 with > `apt-get full-upgrade`. There were errors involving upgrading > postgresql and bacula, seemingly because dpkg tried to configure > packages in the wrong order. A few iterations eventually led to no > further errors from apt-get or dpkg. > > I had to manually run the update_postgres_tables script, and that > required modding the script to correct for the fact that bash will > convert "$$" into the current PID, breaking psql if-blocks. > > Bacula no longer complains about database versions (found 1019, want > 1026, etc), but now all my backup jobs fail at the end with the > following log message: > > JobId 48: Fatal error: backup.c:343 [DE0028] Query failed: DECLARE > _bac_cursor CURSOR FOR SELECT Path, Filename, FileIndex, JobId, LStat, > DeltaSeq , JobTDate FROM ((SELECT Path.Path, T1.Filename, > T1.FileIndex, T1.JobId, LStat, DeltaSeq , JobTDate FROM ( SELECT > DISTINCT ON (Filename, PathId) JobTDate, JobId, FileId, FileIndex, > PathId, Filename, LStat , DeltaSeq FROM (SELECT FileId, JobId, > PathId, Filename, FileIndex, LStat , DeltaSeq FROM File WHERE JobId > IN (1,23) UNION ALL SELECT File.FileId, File.JobId, PathId, Filename, > File.FileIndex, LStat , DeltaSeq FROM BaseFiles JOIN File USING > (FileId) WHERE BaseFiles.JobId IN (1,23) ) AS T JOIN Job USING (JobId) > ORDER BY Filename, PathId, JobTDate DESC ) AS T1 JOIN Path ON > (Path.PathId = T1.PathId) WHERE FileIndex > 0 ) ) AS U1 ORDER BY > JobTDate, FileIndex ASC: ERR=ERROR: column "filename" does not exist > LINE 1: ... , DeltaSeq FROM (SELECT FileId, JobId, PathId, Filename, ... > > Googling gives results that talk about fixing SQL files that don't exist > on my system and/or do not contain the quoted query. > > All tips on what to do next will be greatly appreciated. > > Bacula version 15.0.3 as packaged for Debian Trixie (13). > Postgresql version 13. > > Thanks, > -Brian > > > _______________________________________________ > Bacula-users mailing list > Bac...@li... > https://lists.sourceforge.net/lists/listinfo/bacula-users > |
|
From: Arno L. <al...@it...> - 2026-11-10 18:48:40
|
Hi Lloyd, all, Am 10.11.2026 um 16:50 schrieb Lloyd Brown via Bacula-users: > It's true that the majority of the VFs were never restored from, but > some certainly were. > > This really gets to the crux of my original question. Why is the cycle > bad? Suboptimal performance, perhaps, though I'd argue that for disks > the difference is negligible. We stumbled over that because performance was so bad that users actually couldn't complete their copy and virtual backups in decent time, couldn't saturate their tape drives, and found that SD threads, doing such jobs, wasted more than 90% (iirc) of a CPUs time going through bootstrap records. Negligible only if you accepted a whole university's backups to simply not take place any more :-) > But what makes it bad/incorrect in some > way? That's what I've been trying to understand. Optimizing bootstrap record handling in the way it was implemented required removing the ability to revisit volumes once processed. I'm pretty sure it could have been done differently. I'm also pretty sure that balancing effort, impact and potential risk of new bugs was done and the result was what we have now. You should find the comment /* * ***FIXME*** * Also for efficiency, once a bsr is done, it really should be * delinked from the bsr chain. This will avoid the above * problem and make traversal of the bsr chain more efficient. * * To be done ... */ in the match-bsr.c source file of the SD, which indicates the developers could imagine alternative solutions, as well as #define new_match_all match_all /* Code to test the new match_all code. Can be disabled with a setdebug command */ int use_new_match_all = 0; which indicates you could probably even test the impact, although you'd probably have to dig out and modify the, by now, probably hard-coded DIR code that prevents the "loops" to be created in bsr files. I would assume that Eric and team would be considering a robust implementation that raises the existing limitation. Some of our useres surely would appreciate that. There are really critical code paths involved, though. Cheers, Arno -- Arno Lehmann IT-Service Lehmann Sandstr. 6, 49080 Osnabrück |