modding HUGE Problem with Modding RDR2 That No One is Talking About

[RLTYProds 1]

Thank you, got it working! Thanks for making solutions to this problem, hopefully we get some from Rockstar themselves too, but I ain't holding my breath.

[Edegon 1]

2 hours ago, werwolf969 said:

Since this code not for Python but for , as I specified on Pastebin, best bet will be to save script code as .ps1 file.

 

 

I see, thank you.

I think deadwoodd made a mistake here then:

11 hours ago, deadwoodd said:

which is where the Python script comes into play.

 

[werwolf969 27]

43 minutes ago, RLTYProds said:

Thanks for making solutions to this problem

You're welcome, but I wouldn't call them solutions. Because the root cause of game logic breaks is much more complex than simply inserting files into game archives with different method. When the situation is sorted out, we can use streaming again, and I think all this will be redundant.

Looking at the GTA V game configuration and additional ASI plugins, manipulating in-game memory limits and pools, I think that is where we should look. And maybe, after that, LML for RDR2 will support adding external RPF archives with modified data inside the game, which will solve this problem.

[Edegon 1]

2 minutes ago, werwolf969 said:

When the situation is sorted out, we can use streaming again, and I think all this will be redundant.

dominatorgt said that:

Quote

File Streaming:

1. Process: In file streaming, the file is loaded into memory in chunks rather than the entire file at once. This means that only the parts of the file that are needed at a given time are loaded into memory
2. This can be taxing for SSDs, as they are not as efficient at handling small files
3.  

• Frequent Random Access: If rapplication constantly jumps around to different small files on the SSD, then it can be less efficient.
• Many Temporary Files: If streaming creates a large number of temporary small files that are constantly written and deleted, it can impact SSD lifespan over time.

Here.
Do you think points 2 and 3 can be fixed too?

[werwolf969 27]

Just now, Edegon said:

Do you think points 2 and 3 can be fixed too?

I can't say anything until we get RPF archive support in LML for RDR2. It will depend on how the data from them is inserted into the game. Currently, any new files you add to the game will be streamed (there is nothing to replace when adding new content). Therefore, all the thousands of clothing texture dictionaries from WhyEm/EEE/GFA, all additional YMAP locations, and all new YMT definitions for newly added content will be streamed into the game through LML. You can use replacements (and many mods do that) for existing files, and this surely has less strain on your storage and on the game engine, as DominatorGT mentions in his posts. However, this will not solve the problem of newly added content, and for me, this is what matters the most.

How external RPF support could possibly work, I don't know. I'm too lazy to install GTA V, start modding it, and search for answers.

[Edegon 1]

1 hour ago, werwolf969 said:

Here's an example of this file. You can paste any path there, including the path to any file in the game archive, even if it is not in the source directory (making "source paths" a cheat sheet for the script). The script will only choose paths to files in the mod directory. However, for performance reasons, it's best to filter the data in this file to something similar to the source directory content.

source_paths.txt 322.34 kB · 4 downloads

So I need to manually find, copy and paste the original path of every file in the mod from OpenIV and paste it there?

Sometimes the file is in more paths, all of them (excluding the one of the mod itself) must be put in source_paths.txt?

Edited Tuesday at 12:39 PM by Edegon

[werwolf969 27]

*doubleposted*

Edited Tuesday at 12:42 PM by werwolf969

[werwolf969 27]

11 minutes ago, Edegon said:

So I need to manually find, copy and paste the original path of every file in the mod from OpenIV and paste it there?

Correct.

 

11 minutes ago, Edegon said:

Sometimes the file is in more paths, I guess I should choose just the one of update.rpf, right?

Just copy them all. The script will insert a path with LML mount points for each of them if there is a file with the same name in the source folder.

In fact, for even easier "non-brainer" install.xml file generation, you should use this version of the script.

https://pastebin.com/ZyZePbjX

A made changes. So, if there is more than one mount point for a source file in LML, the script will add the "allowDuplicate=true" option to the node. Thus, LML will replace the file in every archive within the game. And you will be sure that you actually loaded the replacement file into the game.

You can search for files not by their full name, but by some similar part of their name. For example, if you have files to replace with names like:
crp_berry.ytd

crp_berry_aa_sim.yft

crp_berry_har_aa_sim.yft

 

You can search for crp_berry and copy all the paths into the source_paths. The script will choose only existing files from the source folder to paste into install.xml.

Edited Tuesday at 12:46 PM by werwolf969

[deadwoodd 1]

8 hours ago, Edegon said:

 

I see, thank you.

I think deadwoodd made a mistake here then:

 

Can confirm, I am indeed an idiot 😁 I'll edit the post to fix that and will clarify it's a powershell script, not python

[BitTitan 0]

@werwolf969 -- Dude you are a hero among men 😄  

 

I'm looking at modifying your updated (Duplicates=True) version so that it also outputs to /nested/game/path/folder/moddedfile.ext.txt.   Reasoning for this is I use ModOrganizr to manage RDR mods.  After over a decade of Bethesda modding, I'm spoiled in what a mod manager should do.  I'm still limited by what RDR requires to inject modded files, but your scripts and masochist methods of managing modded files is working well -- I think.  Because I'm choose to not rely on LMM to handing conflict detection; frankly I'm not even sure LMM has conflict detection outside of looking the log file for OVERWRITE.

 

My PowerShell-fu is weak and I'm converting Linux shell logic into PowerShell, but I'll get there soon as I figure out the translation and logic differences.  My napkin logic (which at this point looks like the broken English of coding) is:

mkdir -p nested\${RemainingPath}\${MatchingFile}.txt

I'm not 100% on which variables make the most sense and obviously that syntax is now PowerShell.

 

 

[werwolf969 27]

Thank you for your kind words. I'm just trying to be helpful to the community.

 

1 hour ago, BitTitan said:

LMM has conflict detection

LMM/LML does not have any conflict detection. It will overwrite files with the same name, choosing only by their order in the mods.xml file inside the LML folder. If you enable the overwrite flag for a mod, then the LML will load this mod files exclusively last. And if you have two mods with the overwrite flag enabled, and they have same file to load, then it will load the last one in the load order.

So, to be sure you haven’t messed up your load order, it’s best to make sure you don’t have any of the same files in any mods. Do a manual merge for the same files, put them in a separate folder, create an install.xml file for them, and rename the original mod files that you replaced with something.ext_org so LML doesn’t load them at game start.

1 hour ago, BitTitan said:

mkdir -p nested\${RemainingPath}\${MatchingFile}.txt

I'm not sure if I fully understand the problem you are trying to solve. However, I assume that you want to create an .xml file and then upload all the files from it to a separate folder, like making a backup or something? If you want, PM me, and we'll try to figure something out.

Edited 7 hours ago by werwolf969

[TheCubanMissile2 0]

I've been doing some testing and have the train blimps mod installed. After completing a mission a bunch of blimps get removed from 9 to 3 or 4. Anyone else experience this? I have my lml mods disabled for now and a couple asi mods.

[JKL409 8]

Did this original issue ever get fixed or was progress made on it...? I'm unclear as I've not been checking the thread properly for a few months now. 

[xerofilontoXerofilonto 1]

Hello, this research on trying to combat the problem is amazing but we need to present it in a more straighforward way maybe in other post or something so that people who want to patch out their own texture mods can do so without scrolling various pages of the forum for nesessary information i myself i'm very confused but i haven't had time to read all the comments in this post

[werwolf969 27]

4 hours ago, JKL409 said:

Did this original issue ever get fixed

This problem has not been fixed since the RAGE engine mods were introduced, only partially solved through workarounds such as expanding memory through third-party plugins and packaging mods in RPF archives that are understandable to the engine. For GTA V, it took several years until such tools were created, the RDR 2 community is an order of magnitude smaller, so everything is done longer here. Perhaps the recent release of the RDR remaster will help speed up the appearance of such tools.

 

1 hour ago, xerofilontoXerofilonto said:

more straighforward way

So far, there are guesses posted here, some attempts to solve the problem, but there is no solution. It will be correct to wait for the solution to appear, and I am sure it will be issued as a separate mod on all sites.