Loading [MathJax]/extensions/Safe.js

Special Aircraft Service

Individual Mods and Packs for IL-2 1946 => Other Mods => ModWorx => Topic started by: benitomuso on February 11, 2020, 04:39:03 AM

Title: Total MODder V4 and SFS Manager V4 [4.09] [4.10] [4.11] [4.12]
Post by: benitomuso on February 11, 2020, 04:39:03 AM
Total MODder V4 and SFS Manager V4


After having some short time in between my new father role and tens of professional duties I have had the time to play some little Il-2. I needed to upgraded the TotalMODder to the v4 with several bugfixes and minor improvements, specially considering the new WAW3, DOF3, etc folders.

At the same time I removed all restrictions from the SFS Manager for this V4, so now massive SFS packing and other features are all unrestricted.


For those who didn't know the Tools I would only suggest going to the previous versions posts and Documentation:

Original TotalMODder V3 Easy Pack post by PAL (https://www.sas1946.com/main/index.php/topic,42295.0.html)


Former Documentation:

TotalMODder V3 ReadMe by PAL (https://www.mediafire.com/view/tjpjmals9j6g8rb/il2fbMODder-ReadMe.pdf)




Download TotalModder v4, from here (https://www.mediafire.com/file/y7632c9w89bvm4t/il2fbMODderV4.rar/file)
(Updated and bugfixed to work with BAT, etc)

(https://i.postimg.cc/CLS9Dp7t/Il2-MODder.png) (https://postimages.org/)



Download SFSManager v4, from here (https://www.mediafire.com/file/ee0puxxidnpxiqm/SFSManagerV4.rar/file)
(Completely unrestricted with all features available)

(https://i.postimg.cc/J4rd0c4C/SFSManager.png) (https://postimages.org/)


Regards, Pablo

Download link fo version 4.1 with additional bugfixes (see post #17) https://www.mediafire.com/file/e0ovn0jcm1jrbta/SFSManager_V4.1.rar/file
(link courtesy of SSN637 post#20)

Title: Re: Total MODder V4 and SFS Manager V4
Post by: SAS~Storebror on February 11, 2020, 04:45:40 AM
Very cool Pablo, thanks so much!
Now if only we could pack our own files.SFS, that's the single thing that remains...

]cheers[
Mike
Title: Re: Total MODder V4 and SFS Manager V4 [4.09] [4.10] [4.11] [4.12]
Post by: whistler on February 11, 2020, 04:52:44 AM
Yes! A comeback with a BANG! ]hello2[
Title: Re: Total MODder V4 and SFS Manager V4 [4.09] [4.10] [4.11] [4.12]
Post by: erafitti on February 11, 2020, 05:59:28 AM
Welcome back from a v3beta user. It is the best mod manager.  ]wav[

Un saludo.
Title: Re: Total MODder V4 and SFS Manager V4 [4.09] [4.10] [4.11] [4.12]
Post by: Epervier on February 11, 2020, 07:30:26 AM
Thank you very much !

But I have a big problem with SFS Manager v4.
I tried to load the old file "SFSExtract.lst" (FileList/Database Import/...) which is 46Mb.
After loading I quit the program. I am offered to save the new database.
And I find it corrupted (SFSManager.lst) making 14Mo instead of the initial 20Mo.
And impossible to open a standard SFS file.
Any attempt to modify the database (SFSManager.lst) is doomed to failure.

Same thing if I do "FileList/FileList DataBase Clean/Order" !

I solved the problem by doing the manipulation in version 3.1!
Title: Re: Total MODder V4 and SFS Manager V4
Post by: benitomuso on February 11, 2020, 10:03:57 AM
Thank you very much !

But I have a big problem with SFS Manager v4.
I tried to load the old file "SFSExtract.lst" (FileList/Database Import/...) which is 46Mb.
After loading I quit the program. I am offered to save the new database.
And I find it corrupted (SFSManager.lst) making 14Mo instead of the initial 20Mo.
And impossible to open a standard SFS file.
Any attempt to modify the database (SFSManager.lst) is doomed to failure.

Same thing if I do "FileList/FileList DataBase Clean/Order" !

I solved the problem by doing the manipulation in version 3.1!


Oh, I didn't know about this issue. I will take a look at it, but I don't know when.

Regards, Pablo
Title: Re: Total MODder V4 and SFS Manager V4 [4.09] [4.10] [4.11] [4.12]
Post by: Piotrek1 on February 11, 2020, 10:14:16 AM
Quote
After having some short time in between my new father role and tens of professional duties I have had the time to play some little Il-2. I needed to upgraded the TotalMODder to the v4 with several bugfixes and minor improvements, specially considering the new WAW3, DOF3, etc folders.
Congratulations Pablo, this is great news 8)... and new challenges :), I wish you and your family all the best in your private and professional life,
Thank you also for TMDder, whose older version I've been using since the beginning.

P.s. I have also a question about this mod:
https://www.sas1946.com/main/index.php?topic=24777.0
Are you planning any update?

Best regards,
Piotrek
Title: Re: Total MODder V4 and SFS Manager V4
Post by: benitomuso on February 12, 2020, 05:45:13 AM
Quote
After having some short time in between my new father role and tens of professional duties I have had the time to play some little Il-2. I needed to upgraded the TotalMODder to the v4 with several bugfixes and minor improvements, specially considering the new WAW3, DOF3, etc folders.
Congratulations Pablo, this is great news 8)... and new challenges :), I wish you and your family all the best in your private and professional life,
Thank you also for TMDder, whose older version I've been using since the beginning.

P.s. I have also a question about this mod:
https://www.sas1946.com/main/index.php?topic=24777.0
Are you planning any update?

Best regards,
Piotrek


Hi Piotrek.

Regarding the AutoDiffFM+, as far as I remember it was included in the EngineMod a long time ago by Anto. I think the current versions of Western should have kept it but I cannot be sure, because the evolution of the EngineMod was always very unsteady and subjective.

For sure I'm not going to update it. For any Java coder merging what I used in that mod that was sequentially testing the different known encodings of buttons up to finding the right one is a 10 min task.

Are you sure the "current" EngineMod doesn't load buttons from any version?

Regards, Pablo

Title: Re: Total MODder V4 and SFS Manager V4 [4.09] [4.10] [4.11] [4.12]
Post by: Piotrek1 on February 12, 2020, 07:08:45 AM
Hello Pablo, Thank you for answering, I asked out of curiosity, because this mod has always did the job when I was on DBW, almost all the planes for 4.09 I could use without any problems, your answer is exhausting my question and is enough for me, Thank you.
Best Regards,
Piotrek
Title: Re: Total MODder V4 and SFS Manager V4 [4.09] [4.10] [4.11] [4.12]
Post by: vonOben on March 20, 2020, 03:56:49 AM
Many thanks for the update Pablo!  :)

I've been using v3.0 and 3.1 since they were released, excellent program!  :)

Best regards

vonOben
Title: Re: Total MODder V4 and SFS Manager V4 [4.09] [4.10] [4.11] [4.12]
Post by: Knochenlutscher on April 12, 2020, 05:08:07 AM
Cool stuff Pablo, Thanks for these Upgrades.
I love your Tools, they are palin handy and easy to use.

Best wishes
Tobias
Title: Re: Total MODder V4 and SFS Manager V4
Post by: Stainless on April 12, 2020, 11:37:11 AM
Very cool Pablo, thanks so much!
Now if only we could pack our own files.SFS, that's the single thing that remains...

]cheers[
Mike

Do we know the SFS files structure?

Is there a document I could read somewhere?

Title: Re: Total MODder V4 and SFS Manager V4
Post by: benitomuso on April 13, 2021, 09:48:17 AM
Very cool Pablo, thanks so much!
Now if only we could pack our own files.SFS, that's the single thing that remains...

]cheers[
Mike

Do we know the SFS files structure?

Is there a document I could read somewhere?



Hi Stainless, I haven't seen this question.

Yes, I quite detailed found the structure, that's the way I created these tools.

Much help was analyzing the original SFSExtractor of QTim:

https://www.sas1946.com/main/index.php?topic=7017.0

There were the C++ sources somewhere and I started from there.

Essentially they are a pack of files subdivided in blocks, each one being a Deflate encypted one. The blocks are addressed from some sort of "table of contents".

I never had the time nor the will to document it.

Regards, Pablo.

Title: Re: Total MODder V4 and SFS Manager V4 [4.09] [4.10] [4.11] [4.12]
Post by: ssn637 on March 04, 2022, 03:54:25 AM
Pablo

I just wanted to express my sincere gratitude and appreciation for this wonderful tool. 

I now refer to the SFS Manager as the "little program that could" after having extracted (albeit with another SFS Extractor because the SFS Manager extract tab froze in Windows 11 after loading) every SFS archive in my BAT 4.2.1 installation to a single BATMOD folder, eliminating any duplicates from the original (which still came to a whopping 130 GB of data!) and repacking the lot into a batch of split SFS archives (a total of 488 SFS files numbered from BAT00.SFS to BAT487.SFS!).  Now for the amazing part -- it worked! I replaced the BAT00, BAT03 and BAT03B folders with the BATMOD "clean installation" and loaded up the game for a quick spin, all the while telling myself that the 3 hours I'd spent endlessly clicking the "OK" button while the SFS Manager did its thing breaking up that huge folder into hundreds of split archives would surely be futile.  Incredible!

Just a few questions for you if you don't mind my impertinence:
1. Could you please check the extractor portion of the SFS Manager V4 to see if you can replicate my issue?  I am able to load an SFS archive but cannot extract it to a separate folder -- when I click on the "extract all" command nothing happens.
2.  When splitting a folder into separate SFS archives while repacking, is there any way to get around the current limit of 250 MB per package?  This results in SFS archives that are between 60 - 90 MB in size and I'd love to be able to raise that limit to around 1.5 GB to create repacked archives of around 500 MB each.  This would greatly reduce the time needed to repack such a large folder of data.
3.  Would it be possible to run the repack command without needing to click "OK" each time an SFS archive is generated?  That way if I ever choose to challenge myself with such an immense task again I could split and repack the source folder overnight.

Thanks again for all you've done for this community!

Cheers

Geoffrey
Title: Re: Total MODder V4 and SFS Manager V4 [4.09] [4.10] [4.11] [4.12]
Post by: benitomuso on March 04, 2022, 07:59:16 AM
Hi Geoffrey

Regarding your questions, I'm really out of business about MODding.

I doubt I would have time to review your points 1, 2 and 3 in the near future. Nonetheless I'm interested in your repacking of the BAT files? How big is your repacking regarding the original size of the bunch of SFSs?

If you make a clean repack of it with all the contents I would be one of the first interested persons on it.

Let me see, perhaps during the weekend I could open the project and going particularly to those issues you mention.

Regards, Pablo
Title: Re: Total MODder V4 and SFS Manager V4 [4.09] [4.10] [4.11] [4.12]
Post by: Knochenlutscher on March 04, 2022, 11:23:40 AM
Hi Pablo, Hi Geoffrey,

You can split Files with Total Modder? really? I did this manually, crap, I'm dumb.

Yeah, that double click OK, good request,

Pablo? I love this Proggie, it's a pleasure to use and so powerfull. I can't mod/live without.  :)

Best wishes
Tobias
Title: Re: Total MODder V4 and SFS Manager V4 [4.09] [4.10] [4.11] [4.12]
Post by: ssn637 on March 04, 2022, 12:04:59 PM
Hi Pablo

Wow, thanks for your quick response!  I didn't expect any, to tell you the truth...

Of course I can easily understand that you've moved on to bigger and better challenges, but I'm not sure you realize just how many fans you have out there who admire and respect you tremendously for every improvement you've brought us over the years.  Thank you!

Anyway, I suppose I should explain what prompted me to try to "clean up" the SFS archives.  Carsmaster provided us with an awesome upgrade for the default concrete runways that I loved, so I thought it might be a good idea to replace the original 3do/Airfield/Concrete folder with his to make the modification permanent, rather than adding them via JSGME.  I started searching through the BAT00, BAT03 and BAT03B folders and found, to my surprise, that there are no less than nine SFS archives containing the identical file names that needed replacing!  Of course, you and all the experts are well aware of this, since these WAW and BAT archives have evolved over time, but I wondered how often this might be the case for other 3do, effects, hex files etc.  If duplicate files are scattered throughout the SFS archives, then why not clean them up to save space?

After extracting all the SFS archives and retaining only the latest versions of all duplicates I was able to save approximately 15 GB of space.  There were a lot of effects, for example, that I removed by replacing them with WxTech's V1.4 version wherever possible and creating a separate WxTechEffects.SFS archive from the lot.  I also moved any presets or sounds files to my_presets and my_sounds, since every time I launch a mission the log file shows there are presets NOT FOUND! that I then add manually after searching for their equivalents.  There were also folders like "il8n", "gui", "cache" and "paintschemes" that I deleted because they were already in the STD folder or elsewhere.  I ended up with a zillion hex files, a 3do, cod and maps folder that totalled 130 GB.  Then I simply opened this folder in SFS Manager (it took a while!), checked the option to repack them into 250 MB pieces (the max possible) and let the program run to completion.  The resulting 488 SFS archives, plus the WxTechEffects.SFS came in at just under 40 GB total. 

I'm sure I did something wrong and if anyone were to conduct extensive testing would find errors with what I came up with (for example, about a half-dozen maps failed to load, which I solved by replacing their load.ini file from the XUPMOT archves with the BAT originals), but in theory such a cleanup of the SFS archives does offer the advantage of eliminating duplicate entries and a 25% reduction of the BAT 4.2.1 total, give or take. 

I hope I was able to answer your question and, if not, please feel free to let me know if I can be of any help. 

Take care Pablo and best wishes from all of us who have for so long profited from your incredible efforts here!

Geoffrey
Title: Re: Total MODder V4 and SFS Manager V4
Post by: benitomuso on April 01, 2022, 03:11:56 PM
Hi Geoffrey.

Try this update, I think that all what you mentioned is addressed:

https://drive.google.com/file/d/1085m8jb9kRXaiPRoDwDOWmfNfGzENykc/view?usp=sharing (https://drive.google.com/file/d/1085m8jb9kRXaiPRoDwDOWmfNfGzENykc/view?usp=sharing)

Regards, Pablo.

Hi Pablo

Wow, thanks for your quick response!  I didn't expect any, to tell you the truth...

Of course I can easily understand that you've moved on to bigger and better challenges, but I'm not sure you realize just how many fans you have out there who admire and respect you tremendously for every improvement you've brought us over the years.  Thank you!

Anyway, I suppose I should explain what prompted me to try to "clean up" the SFS archives.  Carsmaster provided us with an awesome upgrade for the default concrete runways that I loved, so I thought it might be a good idea to replace the original 3do/Airfield/Concrete folder with his to make the modification permanent, rather than adding them via JSGME.  I started searching through the BAT00, BAT03 and BAT03B folders and found, to my surprise, that there are no less than nine SFS archives containing the identical file names that needed replacing!  Of course, you and all the experts are well aware of this, since these WAW and BAT archives have evolved over time, but I wondered how often this might be the case for other 3do, effects, hex files etc.  If duplicate files are scattered throughout the SFS archives, then why not clean them up to save space?

After extracting all the SFS archives and retaining only the latest versions of all duplicates I was able to save approximately 15 GB of space.  There were a lot of effects, for example, that I removed by replacing them with WxTech's V1.4 version wherever possible and creating a separate WxTechEffects.SFS archive from the lot.  I also moved any presets or sounds files to my_presets and my_sounds, since every time I launch a mission the log file shows there are presets NOT FOUND! that I then add manually after searching for their equivalents.  There were also folders like "il8n", "gui", "cache" and "paintschemes" that I deleted because they were already in the STD folder or elsewhere.  I ended up with a zillion hex files, a 3do, cod and maps folder that totalled 130 GB.  Then I simply opened this folder in SFS Manager (it took a while!), checked the option to repack them into 250 MB pieces (the max possible) and let the program run to completion.  The resulting 488 SFS archives, plus the WxTechEffects.SFS came in at just under 40 GB total. 

I'm sure I did something wrong and if anyone were to conduct extensive testing would find errors with what I came up with (for example, about a half-dozen maps failed to load, which I solved by replacing their load.ini file from the XUPMOT archves with the BAT originals), but in theory such a cleanup of the SFS archives does offer the advantage of eliminating duplicate entries and a 25% reduction of the BAT 4.2.1 total, give or take. 

I hope I was able to answer your question and, if not, please feel free to let me know if I can be of any help. 

Take care Pablo and best wishes from all of us who have for so long profited from your incredible efforts here!

Geoffrey
Title: Re: Total MODder V4 and SFS Manager V4 [4.09] [4.10] [4.11] [4.12]
Post by: ssn637 on April 02, 2022, 08:21:28 AM
Thank you so much Pablo!

All the features I've asked for have been included with Version 4.1 and I'm in the middle of repacking everything again!

I really appreciate the time and effort you've put into this despite your busy schedule. 

Can't wait to see what the resulting SFS archives look like and how much space I'll be able to save along with the elimination of any duplicates!  This time around I'm creating one set of SFS archives from all the Hex files, another from each of the COD, 3do and Maps folders to make it easier to find individual files should I need to update them. 

Thanks again and I'll let you know how it goes when I'm done!

Cheers

Geoffrey


Update : Success! I packed the SFS archives last night, packing the Hex (Class) folder in splitted SFS files of 2000 MB at normal compression for a total of 32 SFS archives, the 3do folder in 1500 MB chunks and the Maps folder in 1000 MB chunks.  The resulting BAT00 folder totalled 39 GB for a 30% reduction compared to the original BAT 4.2.1 combined BAT00, BAT03 and BAT03B folders!  Before packing the lot I removed any unwanted files such as desktop.ini, "old", "backup", "original" and "readme" files, as well as any folders or files beginning with "-" (except for the HEX files located in the COD folder, since I wasn't sure what to do with these so I just left them alone). 

I only noted two issues during the process:
1. When running the SFS Manager V4.1, if you want to extract an SFS archive you must click on the "Extract All" button in the bottom-right corner of the GUI rather than from the drop-down menu at the top, which didn't do anything.
2. I had trouble loading the Guadacanal map (load.ini and eload.ini failing) so I simply removed its folder from the Maps archives sourced from the original BAT and XUPMOT folders, which loaded the original IL2 Maddox map instead without error.

What an incredible little program you've given us, Pablo!  A real gift to this community for which we're surely eternally grateful.  Awesome stuff!
Title: Re: Total MODder V4 and SFS Manager V4 [4.09] [4.10] [4.11] [4.12]
Post by: eMJayCZ on January 21, 2025, 02:00:36 PM
Hello, can you please reupload the 4.1 version? The Google drive link is dead. Thank you!  ]thumbsup[
Title: Re: Total MODder V4 and SFS Manager V4 [4.09] [4.10] [4.11] [4.12]
Post by: ssn637 on January 21, 2025, 10:31:50 PM
I've uploaded SFSManager V4.1 here.  I hope it's OK with Pablo.

https://www.mediafire.com/file/e0ovn0jcm1jrbta/SFSManager_V4.1.rar/file (https://www.mediafire.com/file/e0ovn0jcm1jrbta/SFSManager_V4.1.rar/file)
Title: Re: Total MODder V4 and SFS Manager V4 [4.09] [4.10] [4.11] [4.12]
Post by: eMJayCZ on January 22, 2025, 08:31:03 AM
I've uploaded SFSManager V4.1 here.  I hope it's OK with Pablo.

https://www.mediafire.com/file/e0ovn0jcm1jrbta/SFSManager_V4.1.rar/file (https://www.mediafire.com/file/e0ovn0jcm1jrbta/SFSManager_V4.1.rar/file)

Thanks for the reupload!