mirrors/rt64
1
0
Fork 0
mirror of https://github.com/rt64/rt64.git synced 2025-12-05 01:00:13 +00:00
Code Issues Packages Projects Releases Wiki Activity

Add support for shift configuration on texture packs. Improve behavior for combined texture packs. (#173)

Browse Source
This commit is contained in:
Darío
2025-06-22 20:41:42 -03:00
committed by GitHub
parent 5097320f7b
commit a0a0f94a01
15 changed files with 516 additions and 295 deletions
Download Patch File Download Diff File Expand all files Collapse all files

54
TEXTURE-PACKS.md
View File

@@ -32,8 +32,10 @@ The configuration file (`rt64.json`) follows this format:
{
"configuration": {
"autoPath": "rice",
"configurationVersion": 2,
"hashVersion": 2
"configurationVersion": 3,
"hashVersion": 5,
"defaultOperation": "stream",
"defaultShift": "half"
},
"operationFilters": [
{
@@ -43,6 +45,15 @@ The configuration file (`rt64.json`) follows this format:
...
],
"shiftFilters": [
{
"wildcard": "Terrain/*",
"shift": "none"
},
...
],
"textures": [
{
@@ -73,22 +84,24 @@ The configuration file (`rt64.json`) follows this format:
- **rt64**: The hash from RT64 as a file name. No texture packs use this yet and it's unlikely a texture pack author would prefer to use this instead of naming the files manually.
- **configurationVersion**: The version of the configuration file format.
- **hashVersion**: The version of the hash algorithm used by RT64.
- **defaultOperation**: Determines the default loading operation for all textures in the pack. See the ["Operation"](#operation) section for more details.
- **defaultShift**: Determines the default texture shift behavior for all textures in the pack. See the ["Shift"](#shift) section for more details.
### Operation Filters
### Filters
Operation filters are optional filters that define how textures should be loaded based on their names. This can be handy if your files are well organized, as it's easier to set the operation you want using filters instead of having to add an entry for each file separately.
Filters are optional and can define how textures should be loaded (operations) or how they should be visually shifted (shift) based on their names. This can be handy if your files are well organized, as it's easier to set the options you want using filters instead of having to add an entry for each file separately.
- **wildcard**: Pattern to match texture paths. Use * to indicate any number of characters (including *zero*) and ? to indicate any *one* character. Some examples include:
- "Terrain/Grass001.*" will include any file called "Grass001" inside the directory "Terrain".
- "Text/*" will include any files inside the directory "Text".
- "\*/Button\*" will include any files that start with "Button" inside any directory.
- "UI/Letter?.*" will include any files called Letter with exactly one character that can be anything (LetterA, LetterB, etc.) inside the directory "UI".
- **operation**: Determines the loading behavior. Possible values are:
- **stream**: Textures are loaded asynchronously, potentially causing visual pop-in. This is **solved by generating a low mipmap cache** via the `texture_packer` tool. This is the default behavior.
- **preload**: Textures are loaded at the start of the game and will remain in memory, preventing any sort of pop-in at the expense of memory usage and increased initial loading times. Only recommended for textures where no kind of pop-in is tolerable, even when using the low mipmap cache.
- **stall**: Stops the renderer until the texture is loaded. Can be useful for instances where loading screens are known to appear. **Not recommended**.
- **operation**: Determines the loading behavior. Only possible under the "operationFilters" member. See the ["Operation"](#operation) section for more details.
- **shift**: Determines the texture shift behavior. Only possible under the "shiftFilters" member. See the ["Shift"](#shift) section for more details.
Filters are processed in the order they appear. Keep this in mind to resolve any potential conflicts between the filters. Feel free to ignore this feature until you feel the need to control the loading behavior of the textures in more detail.
Filters are processed in the order they appear. Keep this in mind to resolve any potential conflicts between the filters. **Filters can't override the setting defined in the entry of an individual texture**.
Feel free to ignore this feature until you feel the need to control the loading behavior or the visual shift of the textures in more detail.
### Textures
@@ -101,6 +114,29 @@ Each texture entry can map hashes together as well as an optional file path.
- Both forward and backward slashes are accepted and should work independently of the platform.
- The texture file does not need to follow a particular naming format.
- If empty, the auto path scheme will be used to try to find the file instead.
- **operation**: An optional setting for the loading operation of the texture. If not defined, the texture will use the default operation or one defined by an operation filter. See the ["Operation"](#operation) section for more details.
- **shift**: An optional setting for the shift behavior of the texture. If not defined, the texture will use the default shift or one defined by a shift filter. See the ["Shift"](#shift) section for more details.
### Operation
Operations are how the renderer will know the loading behavior it should use for a texture. Operations can either be set globally through the texture pack's default option, through an operation filter or by defining it in a texture entry.
The possible options are:
- **stream**: Textures are loaded asynchronously, potentially causing visual pop-in. Pop-in is **solved by generating a low mipmap cache** via the `texture_packer` tool. This is the default behavior and what is preferred for best performance.
- **preload**: Textures are loaded at the start of the game and will remain in memory, preventing any sort of pop-in at the expense of memory usage and increased initial loading times. Only recommended for textures where no kind of pop-in is tolerable, even when using the low mipmap cache.
- **stall**: Stops the renderer until the texture is loaded. Can be useful for instances where loading screens are known to appear. **Not recommended**.
### Shift
Texture shift can control whether the texture will be rendered by preserving the original texture coordinates as-is or by compensating with a half-pixel offset while operating on the original texture's dimensions. Texture shifts can either be set globally through the texture pack's default option, through an operation filter or by defining it in a texture entry.
This option is useful to fix mapping errors related to how the asset was crafted. Due to a quirk in the hardware's texture interpolation, all assets must substract half a texel in order to to look correct. However, not all games were developed with this in mind, and will therefore look incorrect if the renderer compensates for it.
This option allows the texture pack author to have more granular control over this and define how the texture must be mapped. **For assets exported from modern tools such as Fast64, it'll always be the case that half-texel compensation is required**.
The possible options are:
- **half**: Applies half-texel compensation. As of the moment this feature was introduced, **this is the new default behavior**.
- **none**: Texture coordinates are used as-is. This is the default legacy behavior for texture packs before this feature was introduced.
## In-Game Replacement

104
src/common/rt64_filesystem_combined.h
View File

@@ -1,104 +0,0 @@
//
// RT64
//
#pragma once
#include "rt64_filesystem.h"
#include <unordered_map>
namespace RT64 {
struct FileSystemCombinedIterator : FileSystemIteratorImplementation {
std::unordered_map<std::string, uint32_t>::const_iterator iterator;
FileSystemCombinedIterator(std::unordered_map<std::string, uint32_t>::const_iterator iterator) {
this->iterator = iterator;
}
const std::string &value() override {
return iterator->first;
}
void increment() override {
iterator++;
}
bool compare(const FileSystemIteratorImplementation *other) const override {
const FileSystemCombinedIterator *otherCombinedIt = static_cast<const FileSystemCombinedIterator *>(other);
return (iterator == otherCombinedIt->iterator);
}
};
struct FileSystemCombined : FileSystem {
std::vector<std::unique_ptr<FileSystem>> fileSystems;
std::unordered_map<std::string, uint32_t> pathFileSystemMap;
std::shared_ptr<FileSystemCombinedIterator> endIterator;
FileSystemCombined(std::vector<std::unique_ptr<FileSystem>> &fileSystemsTaken) {
fileSystems = std::move(fileSystemsTaken);
// Build map out of the file systems in order.
for (uint32_t i = 0; i < uint32_t(fileSystems.size()); i++) {
for (const std::string &path : *fileSystems[i]) {
pathFileSystemMap[toForwardSlashes(path)] = i;
}
}
endIterator = std::make_shared<FileSystemCombinedIterator>(pathFileSystemMap.cend());
}
~FileSystemCombined() {};
Iterator begin() const override {
return { std::make_shared<FileSystemCombinedIterator>(pathFileSystemMap.cbegin()) };
}
Iterator end() const override {
return { endIterator };
}
bool load(const std::string &path, uint8_t *fileData, size_t fileDataMaxByteCount) const override {
auto it = pathFileSystemMap.find(path);
if (it != pathFileSystemMap.end()) {
return fileSystems[it->second]->load(path, fileData, fileDataMaxByteCount);
}
else {
return false;
}
}
size_t getSize(const std::string &path) const override {
auto it = pathFileSystemMap.find(path);
if (it != pathFileSystemMap.end()) {
return fileSystems[it->second]->getSize(path);
}
else {
return 0;
}
}
bool exists(const std::string &path) const override {
return pathFileSystemMap.find(path) != pathFileSystemMap.end();
}
std::string makeCanonical(const std::string &path) const override {
auto it = pathFileSystemMap.find(path);
if (it != pathFileSystemMap.end()) {
return fileSystems[it->second]->makeCanonical(path);
}
else {
return std::string();
}
}
const std::vector<std::unique_ptr<FileSystem>> &getFileSystems() const {
return fileSystems;
}
// Takes ownership of the vector of file systems passed through.
static std::unique_ptr<FileSystem> create(std::vector<std::unique_ptr<FileSystem>> &fileSystems) {
return std::make_unique<FileSystemCombined>(fileSystems);
}
};
};

2
src/common/rt64_filesystem_zip.cpp
View File

@@ -69,7 +69,7 @@ namespace RT64 {
}
// Ignore entries that don't start with the base path. Take out the base path out of the filename.
std::string fileName(fileStat.m_filename);
std::string fileName = FileSystem::toForwardSlashes(std::string(fileStat.m_filename));
if (!impl->basePath.empty()) {
if (!startsWith(fileName, impl->basePath)) {
continue;

117
src/common/rt64_replacement_database.cpp
View File

@@ -109,20 +109,43 @@ namespace RT64 {
}
}
ReplacementOperation ReplacementDatabase::resolveOperation(const std::string &relativePath, const std::vector<ReplacementOperationFilter> &filters) {
ReplacementOperation resolution = ReplacementOperation::Stream;
ReplacementOperation ReplacementDatabase::resolveOperation(const std::string &relativePath, ReplacementOperation operation) {
if (operation == ReplacementOperation::Auto) {
ReplacementOperation resolution = config.defaultOperation;
// Resolve all filters in the order they appear in the database.
for (const ReplacementOperationFilter &filter : filters) {
if (checkWildcard(relativePath, filter.wildcard)) {
resolution = filter.operation;
// Resolve all filters in the order they appear in the database.
for (const ReplacementOperationFilter &filter : operationFilters) {
if (checkWildcard(relativePath, filter.wildcard)) {
resolution = filter.operation;
}
}
}
return resolution;
return resolution;
}
else {
return operation;
}
}
ReplacementShift ReplacementDatabase::resolveShift(const std::string &relativePath, ReplacementShift shift) {
if (shift == ReplacementShift::Auto) {
ReplacementShift resolution = config.defaultShift;
// Resolve all filters in the order they appear in the database.
for (const ReplacementShiftFilter &filter : shiftFilters) {
if (checkWildcard(relativePath, filter.wildcard)) {
resolution = filter.shift;
}
}
return resolution;
}
else {
return shift;
}
}
void ReplacementDatabase::resolvePaths(const FileSystem *fileSystem, std::unordered_map<uint64_t, ReplacementResolvedPath> &resolvedPathMap, bool onlyDDS, std::vector<uint64_t> *hashesMissing, std::unordered_set<uint64_t> *hashesToPreload) {
void ReplacementDatabase::resolvePaths(const FileSystem *fileSystem, uint32_t fileSystemIndex, std::unordered_map<uint64_t, ReplacementResolvedPath> &resolvedPathMap, bool onlyDDS, std::vector<uint64_t> *hashesMissing, std::unordered_set<std::string> *pathsToPreload) {
std::unordered_map<std::string, std::string> autoPathMap;
for (const std::string &relativePath : *fileSystem) {
const std::filesystem::path relativePathFs = std::filesystem::u8path(relativePath);
@@ -149,23 +172,22 @@ namespace RT64 {
}
}
std::vector<ReplacementOperationFilter> standarizedFilters = operationFilters;
for (ReplacementOperationFilter &filter : standarizedFilters) {
filter.wildcard = FileSystem::toForwardSlashes(filter.wildcard);
}
auto addResolvedPath = [&](uint64_t hash, const std::string &relativePath) {
auto addResolvedPath = [&](uint64_t hash, const std::string &relativePath, ReplacementOperation operation, ReplacementShift shift) {
// Do not modify the entry if it's already in the map.
if (resolvedPathMap.find(hash) != resolvedPathMap.end()) {
return;
}
ReplacementResolvedPath &resolvedPath = resolvedPathMap[hash];
resolvedPath.fileSystemIndex = fileSystemIndex;
resolvedPath.relativePath = relativePath;
resolvedPath.operation = resolveOperation(relativePath, standarizedFilters);
resolvedPath.originalOperation = operation;
resolvedPath.originalShift = shift;
resolvedPath.resolvedOperation = resolveOperation(relativePath, operation);
resolvedPath.resolvedShift = resolveShift(relativePath, shift);
if ((resolvedPath.operation == ReplacementOperation::Preload) && (hashesToPreload != nullptr)) {
hashesToPreload->insert(hash);
if ((resolvedPath.resolvedOperation == ReplacementOperation::Preload) && (pathsToPreload != nullptr)) {
pathsToPreload->insert(relativePath);
}
};
@@ -185,7 +207,7 @@ namespace RT64 {
if (!canonicalPath.empty() && fileSystem->exists(canonicalPath)) {
if (!onlyDDS || (i == 0)) {
std::string relativePathFinal = FileSystem::toForwardSlashes(canonicalPath);
addResolvedPath(hashRT64, relativePathFinal);
addResolvedPath(hashRT64, relativePathFinal, texture.operation, texture.shift);
}
fileExists = true;
@@ -211,7 +233,7 @@ namespace RT64 {
auto it = autoPathMap.find(searchString);
if (it != autoPathMap.end()) {
uint64_t hashRT64 = ReplacementDatabase::stringToHash(texture.hashes.rt64);
addResolvedPath(hashRT64, it->second);
addResolvedPath(hashRT64, it->second, texture.operation, texture.shift);
}
}
@@ -219,6 +241,18 @@ namespace RT64 {
}
}
void ReplacementDatabase::resolveOperations(std::unordered_map<uint64_t, ReplacementResolvedPath> &resolvedPathMap) {
for (auto &it : resolvedPathMap) {
it.second.resolvedOperation = resolveOperation(it.second.relativePath, it.second.originalOperation);
}
}
void ReplacementDatabase::resolveShifts(std::unordered_map<uint64_t, ReplacementResolvedPath> &resolvedPathMap) {
for (auto &it : resolvedPathMap) {
it.second.resolvedShift = resolveShift(it.second.relativePath, it.second.originalShift);
}
}
uint64_t ReplacementDatabase::stringToHash(const std::string &str) {
return strtoull(str.c_str(), nullptr, 16);
}
@@ -274,15 +308,22 @@ namespace RT64 {
void to_json(json &j, const ReplacementConfiguration &config) {
// Always update the configuration version to the latest one when saving.
ReplacementConfiguration defaultConfig;
j["autoPath"] = config.autoPath;
j["configurationVersion"] = defaultConfig.configurationVersion;
j["autoPath"] = config.autoPath;
j["defaultOperation"] = config.defaultOperation;
j["defaultShift"] = config.defaultShift;
j["hashVersion"] = config.hashVersion;
}
void from_json(const json &j, ReplacementConfiguration &config) {
ReplacementConfiguration defaultConfig;
config.autoPath = j.value("autoPath", defaultConfig.autoPath);
config.configurationVersion = j.value("configurationVersion", 1);
// Shift defaults to none to texture packs made with versions previous to 3, which introduced a different default for shifting.
ReplacementShift defaultShift = (config.configurationVersion < 3) ? ReplacementShift::None : defaultConfig.defaultShift;
config.autoPath = j.value("autoPath", defaultConfig.autoPath);
config.defaultOperation = j.value("defaultOperation", defaultConfig.defaultOperation);
config.defaultShift = j.value("defaultShift", defaultShift);
config.hashVersion = j.value("hashVersion", 1);
}
@@ -302,14 +343,25 @@ namespace RT64 {
}
void to_json(json &j, const ReplacementTexture &texture) {
ReplacementTexture defaultTexture;
j["path"] = texture.path;
j["hashes"] = texture.hashes;
if (texture.operation != defaultTexture.operation) {
j["operation"] = texture.operation;
}
if (texture.shift != defaultTexture.shift) {
j["shift"] = texture.shift;
}
}
void from_json(const json &j, ReplacementTexture &texture) {
ReplacementTexture defaultTexture;
texture.path = j.value("path", defaultTexture.path);
texture.hashes = j.value("hashes", defaultTexture.hashes);
texture.operation = j.value("operation", defaultTexture.operation);
texture.shift = j.value("shift", defaultTexture.shift);
}
void to_json(json &j, const ReplacementOperationFilter &operationFilter) {
@@ -319,22 +371,39 @@ namespace RT64 {
void from_json(const json &j, ReplacementOperationFilter &operationFilter) {
ReplacementOperationFilter defaultFilter;
operationFilter.wildcard = j.value("wildcard", defaultFilter.wildcard);
operationFilter.wildcard = FileSystem::toForwardSlashes(j.value("wildcard", defaultFilter.wildcard));
operationFilter.operation = j.value("operation", defaultFilter.operation);
}
void to_json(json &j, const ReplacementShiftFilter &shiftFilter) {
j["wildcard"] = shiftFilter.wildcard;
j["operation"] = shiftFilter.shift;
}
void from_json(const json &j, ReplacementShiftFilter &shiftFilter) {
ReplacementShiftFilter defaultFilter;
shiftFilter.wildcard = FileSystem::toForwardSlashes(j.value("wildcard", defaultFilter.wildcard));
shiftFilter.shift = j.value("shift", defaultFilter.shift);
}
void to_json(json &j, const ReplacementDatabase &db) {
j["configuration"] = db.config;
j["textures"] = db.textures;
j["operationFilters"] = db.operationFilters;
j["shiftFilters"] = db.shiftFilters;
j["extraFiles"] = db.extraFiles;
}
void from_json(const json &j, ReplacementDatabase &db) {
db.config = j.value("configuration", ReplacementConfiguration());
ReplacementConfiguration defaultConfig;
db.config = j.value("configuration", defaultConfig);
db.textures = j.value("textures", std::vector<ReplacementTexture>());
db.operationFilters = j.value("operationFilters", std::vector<ReplacementOperationFilter>());
db.shiftFilters = j.value("shiftFilters", std::vector<ReplacementShiftFilter>());
db.extraFiles = j.value("extraFiles", std::vector<std::string>());
db.buildHashMaps();
// Update configuration version to the latest one after parsing is over.
db.config.configurationVersion = defaultConfig.configurationVersion;
}
};

55
src/common/rt64_replacement_database.h
View File

@@ -22,28 +22,44 @@ namespace RT64 {
enum class ReplacementOperation {
Preload,
Stream,
Stall
Stall,
Auto,
};
enum class ReplacementShift {
None,
Half,
Auto,
};
enum class ReplacementAutoPath {
RT64,
Rice
Rice,
};
NLOHMANN_JSON_SERIALIZE_ENUM(ReplacementOperation, {
{ ReplacementOperation::Preload, "preload" },
{ ReplacementOperation::Stream, "stream" },
{ ReplacementOperation::Stall, "stall" }
{ ReplacementOperation::Stall, "stall" },
{ ReplacementOperation::Auto, "auto" },
});
NLOHMANN_JSON_SERIALIZE_ENUM(ReplacementShift, {
{ ReplacementShift::None, "none" },
{ ReplacementShift::Half, "half" },
{ ReplacementShift::Auto, "auto" },
});
NLOHMANN_JSON_SERIALIZE_ENUM(ReplacementAutoPath, {
{ ReplacementAutoPath::RT64, "rt64" },
{ ReplacementAutoPath::Rice, "rice" }
{ ReplacementAutoPath::Rice, "rice" },
});
struct ReplacementConfiguration {
ReplacementAutoPath autoPath = ReplacementAutoPath::RT64;
uint32_t configurationVersion = 2;
ReplacementOperation defaultOperation = ReplacementOperation::Stream;
ReplacementShift defaultShift = ReplacementShift::Half;
uint32_t configurationVersion = 3;
uint32_t hashVersion = 5;
};
@@ -55,20 +71,33 @@ namespace RT64 {
struct ReplacementTexture {
std::string path;
ReplacementHashes hashes;
ReplacementOperation operation = ReplacementOperation::Auto;
ReplacementShift shift = ReplacementShift::Auto;
bool isEmpty() const {
return hashes.rt64.empty();
}
};
struct ReplacementOperationFilter {
struct ReplacementFilter {
std::string wildcard;
};
struct ReplacementOperationFilter : ReplacementFilter {
ReplacementOperation operation = ReplacementOperation::Stream;
};
struct ReplacementShiftFilter : ReplacementFilter {
ReplacementShift shift = ReplacementShift::Half;
};
struct ReplacementResolvedPath {
uint32_t fileSystemIndex = 0;
std::string relativePath;
ReplacementOperation operation = ReplacementOperation::Stream;
ReplacementOperation resolvedOperation = ReplacementOperation::Auto;
ReplacementOperation originalOperation = ReplacementOperation::Auto;
ReplacementShift resolvedShift = ReplacementShift::Auto;
ReplacementShift originalShift = ReplacementShift::Auto;
};
struct ReplacementMipmapCacheHeader {
@@ -85,6 +114,7 @@ namespace RT64 {
ReplacementConfiguration config;
std::vector<ReplacementTexture> textures;
std::vector<ReplacementOperationFilter> operationFilters;
std::vector<ReplacementShiftFilter> shiftFilters;
std::vector<std::string> extraFiles;
std::unordered_map<uint64_t, uint32_t> tmemHashToReplaceMap;
@@ -93,8 +123,11 @@ namespace RT64 {
ReplacementTexture getReplacement(uint64_t hash) const;
ReplacementTexture getReplacement(const std::string &hash) const;
void buildHashMaps();
ReplacementOperation resolveOperation(const std::string &relativePath, const std::vector<ReplacementOperationFilter> &filters);
void resolvePaths(const FileSystem *fileSystem, std::unordered_map<uint64_t, ReplacementResolvedPath> &resolvedPathMap, bool onlyDDS, std::vector<uint64_t> *hashesMissing = nullptr, std::unordered_set<uint64_t> *hashesToPreload = nullptr);
ReplacementOperation resolveOperation(const std::string &relativePath, ReplacementOperation operation);
ReplacementShift resolveShift(const std::string &relativePath, ReplacementShift shift);
void resolvePaths(const FileSystem *fileSystem, uint32_t fileSystemIndex, std::unordered_map<uint64_t, ReplacementResolvedPath> &resolvedPathMap, bool onlyDDS, std::vector<uint64_t> *hashesMissing = nullptr, std::unordered_set<std::string> *pathsToPreload = nullptr);
void resolveOperations(std::unordered_map<uint64_t, ReplacementResolvedPath> &resolvedPathMap);
void resolveShifts(std::unordered_map<uint64_t, ReplacementResolvedPath> &resolvedPathMap);
static uint64_t stringToHash(const std::string &str);
static std::string hashToString(uint32_t hash);
static std::string hashToString(uint64_t hash);
@@ -112,6 +145,8 @@ namespace RT64 {
extern void from_json(const json &j, ReplacementTexture &texture);
extern void to_json(json &j, const ReplacementOperationFilter &operationFilter);
extern void from_json(const json &j, ReplacementOperationFilter &operationFilter);
extern void to_json(json &j, const ReplacementShiftFilter &shiftFilter);
extern void from_json(const json &j, ReplacementShiftFilter &shiftFilter);
extern void to_json(json &j, const ReplacementDatabase &db);
extern void from_json(const json &j, ReplacementDatabase &db);
};
};

41
src/gui/rt64_debugger_inspector.cpp
View File

@@ -248,17 +248,19 @@ namespace RT64 {
}
};
auto showHashAndReplaceButton = [&](uint64_t replacementHash) {
auto showHashAndReplaceButton = [&](uint64_t replacementHash, ReplacementShift shift) {
char hexStr[64];
snprintf(hexStr, sizeof(hexStr), "%016" PRIx64, replacementHash);
ImGui::Text("Hash 0x%s", hexStr);
ImGui::Text("Hash: 0x%s", hexStr);
ImGui::SameLine();
bool directoryMode = textureCache.textureMap.replacementMap.fileSystemIsDirectory;
const ReplacementDatabase &replacementDb = textureCache.textureMap.replacementMap.directoryDatabase;
if (ImGui::Button("Replace")) {
if (!textureCache.textureMap.replacementMap.fileSystemIsDirectory) {
if (!directoryMode) {
ImGui::OpenPopup(ReplaceDirectoryOnlyModalId);
}
else if (textureCache.textureMap.replacementMap.directoryDatabase.config.hashVersion < TMEMHasher::CurrentHashVersion) {
else if (replacementDb.config.hashVersion < TMEMHasher::CurrentHashVersion) {
ImGui::OpenPopup(ReplaceOutdatedModalId);
}
else {
@@ -267,7 +269,7 @@ namespace RT64 {
std::filesystem::path directoryPath = textureCache.textureMap.replacementMap.replacementDirectories.front().dirOrZipPath;
std::filesystem::path relativePath = std::filesystem::relative(textureFilename, directoryPath);
if (!relativePath.empty()) {
textureCache.addReplacement(replacementHash, relativePath.u8string());
textureCache.addReplacement(replacementHash, relativePath.u8string(), shift);
}
else {
ImGui::OpenPopup(ReplaceErrorModalId);
@@ -311,6 +313,28 @@ namespace RT64 {
ImGui::EndPopup();
}
ReplacementResolvedPath resolvedPath;
if (textureCache.textureMap.replacementMap.getResolvedPathFromHash(replacementHash, replacementDb.config.hashVersion, resolvedPath)) {
ImGui::BeginDisabled(!directoryMode);
int shift = int(resolvedPath.originalShift);
int operation = int(resolvedPath.originalOperation);
ImGui::Text("Replacement: %s", resolvedPath.relativePath.c_str());
if (ImGui::Combo("Shift", &shift, "None\0Half\0Auto\0")) {
textureCache.setReplacementShift(replacementHash, ReplacementShift(shift));
}
if (ImGui::Combo("Operation", &operation, "Preload\0Stream\0Stall\0Auto\0")) {
textureCache.setReplacementOperation(replacementHash, ReplacementOperation(operation));
}
ImGui::EndDisabled();
if (!directoryMode) {
ImGui::Text("Properties can only be edited when loading a texture pack as a single directory.");
}
}
};
outCreateDrawCallKey = false;
@@ -671,7 +695,7 @@ namespace RT64 {
}
if (headerOpen) {
showHashAndReplaceButton(spriteCommand.replacementHash);
showHashAndReplaceButton(spriteCommand.replacementHash, ReplacementShift::Auto);
}
ImGui::PopID();
@@ -1183,7 +1207,10 @@ namespace RT64 {
}
}
else {
showHashAndReplaceButton(callTile.tmemHashOrID);
bool notCopyMode = (callDesc.otherMode.cycleType() != G_CYC_COPY);
bool notPointFiltered = (callDesc.otherMode.textFilt() != G_TF_POINT);
ReplacementShift shift = notCopyMode && notPointFiltered ? ReplacementShift::Auto : ReplacementShift::None;
showHashAndReplaceButton(callTile.tmemHashOrID, shift);
}
uint32_t textureIndex = 0;

2
src/hle/rt64_application.cpp
View File

@@ -293,7 +293,7 @@ namespace RT64 {
// Create the texture cache.
const uint32_t textureCacheThreads = std::max(threadsAvailable / 4U, 1U);
textureCache = std::make_unique<TextureCache>(textureDirectWorker.get(), textureCopyWorker.get(), textureCacheThreads, shaderLibrary.get(), userConfig.developerMode);
textureCache = std::make_unique<TextureCache>(textureDirectWorker.get(), textureCopyWorker.get(), textureCacheThreads, shaderLibrary.get());
// Compute the approximate pool for texture replacements from the dedicated video memory.
const uint64_t MinimumTexturePoolSize = 512 * 1024 * 1024;

27
src/hle/rt64_state.cpp
View File

@@ -2180,6 +2180,33 @@ namespace RT64 {
ImGui::Text("Texture replacement path: %s", replacementPath.c_str());
}
if (!ext.textureCache->textureMap.replacementMap.replacementDirectories.empty()) {
bool directoryMode = ext.textureCache->textureMap.replacementMap.fileSystemIsDirectory;
ImGui::BeginDisabled(!directoryMode);
ReplacementDatabase &replacementDb = ext.textureCache->textureMap.replacementMap.directoryDatabase;
int defaultShift = int(replacementDb.config.defaultShift);
if (ImGui::Combo("Default Shift", &defaultShift, "None\0Half\0")) {
ext.textureCache->setReplacementDefaultShift(ReplacementShift(defaultShift));
}
int defaultOperation = int(replacementDb.config.defaultOperation);
if (ImGui::Combo("Default Operation", &defaultOperation, "Preload\0Stream\0Stall\0")) {
ext.textureCache->setReplacementDefaultOperation(ReplacementOperation(defaultOperation));
}
if (replacementDb.config.defaultOperation != ReplacementOperation::Stream) {
ImGui::Text("Operation modes other than 'Stream' are not good for performance.");
ImGui::Text("Only set this if you know what you're doing!");
}
ImGui::EndDisabled();
if (!directoryMode) {
ImGui::Text("Properties can only be edited when loading a texture pack as a single directory.");
}
}
ImGui::BeginChild("##textureReplacements", ImVec2(0, -64));
ImGui::EndChild();

6
src/render/rt64_framebuffer_renderer.cpp
View File

@@ -277,7 +277,8 @@ namespace RT64 {
uint32_t textureIndex = 0;
bool textureReplaced = false;
bool hasMipmaps = false;
textureCache->useTexture(callTile.tmemHashOrID, submissionFrame, textureIndex, gpuTile.tcScale, gpuTile.textureDimensions, textureReplaced, hasMipmaps);
bool shiftedByHalf = false;
textureCache->useTexture(callTile.tmemHashOrID, submissionFrame, textureIndex, gpuTile.tcScale, gpuTile.textureDimensions, textureReplaced, hasMipmaps, shiftedByHalf);
// Describe the GPU tile for a regular texture.
gpuTile.ulScale.x = gpuTile.tcScale.x;
@@ -290,6 +291,7 @@ namespace RT64 {
gpuTile.flags.fromCopy = false;
gpuTile.flags.rawTMEM = !textureReplaced && callTile.rawTMEM;
gpuTile.flags.hasMipmaps = hasMipmaps;
gpuTile.flags.shiftedByHalf = shiftedByHalf;
}
}
}
@@ -1924,4 +1926,4 @@ namespace RT64 {
unsigned int im3dVertexCount;
};
};
*/
*/

6
src/render/rt64_texture.h
View File

@@ -4,6 +4,7 @@
#pragma once
#include "common/rt64_load_types.h"
#include "rhi/rt64_render_interface.h"
namespace RT64 {
@@ -14,10 +15,11 @@ namespace RT64 {
RenderFormat format = RenderFormat::UNKNOWN;
uint32_t width = 0;
uint32_t height = 0;
uint32_t tlut = 0;
LoadTile loadTile;
uint32_t mipmaps = 0;
uint64_t memorySize = 0;
// These are only stored if developer mode is enabled.
std::vector<uint8_t> bytesTMEM;
bool decodeTMEM = false;
};
};

329
src/render/rt64_texture_cache.cpp
View File

@@ -11,7 +11,6 @@
// Needs to be included before other headers.
#include "gbi/rt64_f3d.h"
#include "common/rt64_filesystem_combined.h"
#include "common/rt64_filesystem_directory.h"
#include "common/rt64_filesystem_zip.h"
#include "common/rt64_load_types.h"
@@ -25,6 +24,8 @@
namespace RT64 {
// ReplacementMap
static_assert(TMEMHasher::CurrentHashVersion < (sizeof(ReplacementMap::resolvedPathMaps) / sizeof(decltype(*ReplacementMap::resolvedPathMaps))), "Path maps must be bigger than the current hash version.");
static const interop::float2 IdentityScale = { 1.0f, 1.0f };
static const uint32_t TextureDataPitchAlignment = 256;
@@ -53,10 +54,14 @@ namespace RT64 {
evictedTextures.emplace_back(it.second.texture);
}
for (auto &map : resolvedPathMaps) {
map.clear();
}
loadedTextureMap.clear();
loadedTextureReverseMap.clear();
unusedTextureList.clear();
resolvedPathMap.clear();
resolvedHashVersions.clear();
lowMipCacheTextures.clear();
replacementDirectories.clear();
@@ -73,7 +78,7 @@ namespace RT64 {
// Erase from the maps and the relative path set.
auto it = loadedTextureReverseMap.find(lastUnusedTexture);
streamRelativePathSet.erase(it->second->second.relativePath);
fileSystemStreamSets[it->second->second.fileSystemIndex].erase(it->second->second.relativePath);
loadedTextureMap.erase(it->second);
loadedTextureReverseMap.erase(it);
@@ -99,6 +104,7 @@ namespace RT64 {
std::vector<ReplacementTexture> newTextures;
for (const ReplacementTexture &texture : directoryDatabase.textures) {
uint64_t rt64 = ReplacementDatabase::stringToHash(texture.hashes.rt64);
auto &resolvedPathMap = resolvedPathMaps[directoryDatabase.config.hashVersion];
auto pathIt = resolvedPathMap.find(rt64);
// Only consider for removal if the entry has no assigned path.
@@ -119,9 +125,9 @@ namespace RT64 {
directoryDatabase.buildHashMaps();
}
bool ReplacementMap::getResolvedPathFromHash(uint64_t tmemHash, ReplacementResolvedPath &resolvedPath) const {
auto pathIt = resolvedPathMap.find(tmemHash);
if (pathIt != resolvedPathMap.end()) {
bool ReplacementMap::getResolvedPathFromHash(uint64_t dbHash, uint32_t dbHashVersion, ReplacementResolvedPath &resolvedPath) const {
auto pathIt = resolvedPathMaps[dbHashVersion].find(dbHash);
if (pathIt != resolvedPathMaps[dbHashVersion].end()) {
resolvedPath = pathIt->second;
return true;
}
@@ -129,10 +135,11 @@ namespace RT64 {
return false;
}
void ReplacementMap::addLoadedTexture(Texture *texture, const std::string &relativePath, bool referenceCounted) {
uint64_t pathHash = hashFromRelativePath(relativePath);
void ReplacementMap::addLoadedTexture(Texture *texture, uint32_t fileSystemIndex, const std::string &relativePath, bool referenceCounted) {
uint64_t pathHash = hashFromRelativePath(fileSystemIndex, relativePath);
assert(loadedTextureMap.find(pathHash) == loadedTextureMap.end());
MapEntry &entry = loadedTextureMap[pathHash];
entry.fileSystemIndex = fileSystemIndex;
entry.relativePath = relativePath;
entry.texture = texture;
@@ -143,8 +150,8 @@ namespace RT64 {
}
}
Texture *ReplacementMap::getFromRelativePath(const std::string &relativePath) const {
uint64_t pathHash = hashFromRelativePath(relativePath);
Texture *ReplacementMap::getFromRelativePath(uint32_t fileSystemIndex, const std::string &relativePath) const {
uint64_t pathHash = hashFromRelativePath(fileSystemIndex, relativePath);
auto it = loadedTextureMap.find(pathHash);
if (it != loadedTextureMap.end()) {
return it->second.texture;
@@ -154,8 +161,12 @@ namespace RT64 {
}
}
uint64_t ReplacementMap::hashFromRelativePath(const std::string &relativePath) const {
return XXH3_64bits(relativePath.data(), relativePath.size());
uint64_t ReplacementMap::hashFromRelativePath(uint32_t fileSystemIndex, const std::string &relativePath) const {
XXH3_state_t xxh3;
XXH3_64bits_reset(&xxh3);
XXH3_64bits_update(&xxh3, &fileSystemIndex, sizeof(uint32_t));
XXH3_64bits_update(&xxh3, relativePath.data(), relativePath.size());
return XXH3_64bits_digest(&xxh3);
}
void ReplacementMap::incrementReference(Texture *texture) {
@@ -209,6 +220,7 @@ namespace RT64 {
for (size_t i = 0; i < textureReplacements.size(); i++) {
if (textureReplacements[i] != nullptr) {
textureReplacements[i] = nullptr;
textureReplacementShiftedByHalf[i] = false;
textureReplacementReferenceCounted[i] = false;
versions[i]++;
}
@@ -232,6 +244,7 @@ namespace RT64 {
cachedTextureDimensions.emplace_back();
textureReplacements.push_back(nullptr);
cachedTextureReplacementDimensions.emplace_back();
textureReplacementShiftedByHalf.emplace_back(false);
textureReplacementReferenceCounted.emplace_back(false);
textureScales.push_back(IdentityScale);
hashes.push_back(0);
@@ -244,6 +257,7 @@ namespace RT64 {
textures[textureIndex] = texture;
cachedTextureDimensions[textureIndex] = interop::float3(float(texture->width), float(texture->height), 1.0f);
textureReplacements[textureIndex] = nullptr;
textureReplacementShiftedByHalf[textureIndex] = false;
textureReplacementReferenceCounted[textureIndex] = false;
textureScales[textureIndex] = IdentityScale;
hashes[textureIndex] = hash;
@@ -255,7 +269,7 @@ namespace RT64 {
listIterators[textureIndex] = accessList.begin();
}
void TextureMap::replace(uint64_t hash, Texture *texture, bool referenceCounted) {
void TextureMap::replace(uint64_t hash, Texture *texture, bool shiftedByHalf, bool referenceCounted) {
const auto it = hashMap.find(hash);
if (it == hashMap.end()) {
return;
@@ -264,6 +278,8 @@ namespace RT64 {
// Do nothing if it's the same texture. If the operations were done correctly, it's not possible for either the texture or the
// replacement to be any different, so the operation can be considered a no-op.
if (texture == textureReplacements[it->second]) {
// The exception is that the shifting mode can be changed in real-time by the user while editing, so we update it regardless.
textureReplacementShiftedByHalf[it->second] = shiftedByHalf;
return;
}
@@ -274,6 +290,7 @@ namespace RT64 {
Texture *replacedTexture = textures[it->second];
textureReplacements[it->second] = texture;
textureReplacementShiftedByHalf[it->second] = shiftedByHalf;
textureReplacementReferenceCounted[it->second] = referenceCounted;
textureScales[it->second] = { float(texture->width) / float(replacedTexture->width), float(texture->height) / float(replacedTexture->height) };
cachedTextureReplacementDimensions[it->second] = interop::float3(float(texture->width), float(texture->height), float(texture->mipmaps));
@@ -286,12 +303,15 @@ namespace RT64 {
}
}
bool TextureMap::use(uint64_t hash, uint64_t submissionFrame, uint32_t &textureIndex, interop::float2 &textureScale, interop::float3 &textureDimensions, bool &textureReplaced, bool &hasMipmaps) {
bool TextureMap::use(uint64_t hash, uint64_t submissionFrame, uint32_t &textureIndex, interop::float2 &textureScale, interop::float3 &textureDimensions, bool &textureReplaced, bool &hasMipmaps, bool &shiftedByHalf) {
textureScale = IdentityScale;
hasMipmaps = false;
shiftedByHalf = false;
// Find the matching texture index in the hash map.
const auto it = hashMap.find(hash);
if (it == hashMap.end()) {
textureIndex = 0;
textureScale = IdentityScale;
return false;
}
@@ -302,11 +322,10 @@ namespace RT64 {
textureScale = textureScales[textureIndex];
textureDimensions = cachedTextureReplacementDimensions[textureIndex];
hasMipmaps = (textureReplacements[textureIndex]->mipmaps > 1);
shiftedByHalf = textureReplacementShiftedByHalf[textureIndex];
}
else {
textureScale = IdentityScale;
textureDimensions = cachedTextureDimensions[textureIndex];
hasMipmaps = false;
}
// Remove the existing entry from the list if it exists.
@@ -353,9 +372,11 @@ namespace RT64 {
// If a texture replacement was used for this texture, decrease the reference in the map.
if (textureReplacementReferenceCounted[textureIndex] && (textureReplacements[textureIndex] != nullptr)) {
replacementMap.decrementReference(textureReplacements[textureIndex]);
textureReplacements[textureIndex] = nullptr;
textureReplacementReferenceCounted[textureIndex] = false;
}
textureReplacements[textureIndex] = nullptr;
textureReplacementShiftedByHalf[textureIndex] = false;
textureReplacementReferenceCounted[textureIndex] = false;
}
// Stop iterating if we reach an entry that has been used in the present.
else if (age == 0) {
@@ -427,7 +448,7 @@ namespace RT64 {
if (!streamDesc.relativePath.empty()) {
ElapsedTimer elapsedTimer;
bool fileLoaded = textureCache->textureMap.replacementMap.fileSystem->load(streamDesc.relativePath, replacementBytes);
bool fileLoaded = textureCache->textureMap.replacementMap.fileSystems[streamDesc.fileSystemIndex]->load(streamDesc.relativePath, replacementBytes);
textureCache->addStreamLoadTime(elapsedTimer.elapsedMicroseconds());
if (fileLoaded) {
@@ -440,7 +461,7 @@ namespace RT64 {
worker->execute();
worker->wait();
textureCache->uploadQueueMutex.lock();
textureCache->streamResultQueue.emplace_back(texture, streamDesc.relativePath, streamDesc.fromPreload);
textureCache->streamResultQueue.emplace_back(texture, streamDesc.fileSystemIndex, streamDesc.relativePath, streamDesc.fromPreload);
textureCache->uploadQueueMutex.unlock();
textureCache->uploadQueueChanged.notify_all();
}
@@ -451,13 +472,12 @@ namespace RT64 {
// TextureCache
TextureCache::TextureCache(RenderWorker *directWorker, RenderWorker *copyWorker, uint32_t threadCount, const ShaderLibrary *shaderLibrary, bool developerMode) {
TextureCache::TextureCache(RenderWorker *directWorker, RenderWorker *copyWorker, uint32_t threadCount, const ShaderLibrary *shaderLibrary) {
assert(directWorker != nullptr);
this->directWorker = directWorker;
this->copyWorker = copyWorker;
this->shaderLibrary = shaderLibrary;
this->developerMode = developerMode;
lockCounter = 0;
@@ -1009,7 +1029,7 @@ namespace RT64 {
// Add the textures to the replacement pool as a loaded texture.
std::unique_lock lock(textureMapMutex);
for (const StreamResult &result : streamResultQueueCopy) {
textureMap.replacementMap.addLoadedTexture(result.texture, result.relativePath, !result.fromPreload);
textureMap.replacementMap.addLoadedTexture(result.texture, result.fileSystemIndex, result.relativePath, !result.fromPreload);
// Increment texture pool memory used permanently if the texture was preloaded.
if (result.fromPreload) {
@@ -1023,11 +1043,12 @@ namespace RT64 {
for (const StreamResult &result : streamResultQueueCopy) {
afterDecodeBarriers.emplace_back(result.texture->texture.get(), RenderTextureLayout::SHADER_READ);
auto it = streamPendingReplacementChecks.find(result.relativePath);
while (it != streamPendingReplacementChecks.end()) {
auto &streamChecks = textureMap.replacementMap.fileSystemStreamChecks[result.fileSystemIndex];
auto it = streamChecks.find(result.relativePath);
while (it != streamChecks.end()) {
replacementQueueCopy.emplace_back(it->second);
streamPendingReplacementChecks.erase(it);
it = streamPendingReplacementChecks.find(result.relativePath);
streamChecks.erase(it);
it = streamChecks.find(result.relativePath);
}
}
}
@@ -1060,15 +1081,15 @@ namespace RT64 {
newTexture->creationFrame = upload.creationFrame;
textureMapAdditions.emplace_back(TextureMapAddition{ upload.hash, newTexture });
if (developerMode) {
newTexture->bytesTMEM = upload.bytesTMEM;
}
newTexture->format = RenderFormat::R8_UINT;
newTexture->width = upload.width;
newTexture->height = upload.height;
newTexture->tmem = copyWorker->device->createTexture(RenderTextureDesc::Texture1D(std::max(uint32_t(upload.bytesTMEM.size()), 1U), 1, newTexture->format));
newTexture->tmem->setName("Texture Cache TMEM #" + std::to_string(TMEMGlobalCounter++));
newTexture->bytesTMEM = upload.bytesTMEM;
newTexture->loadTile = upload.loadTile;
newTexture->tlut = upload.tlut;
newTexture->decodeTMEM = upload.decodeTMEM;
if (!upload.bytesTMEM.empty()) {
void *dstData = tmemUploadResources[i]->map();
@@ -1103,34 +1124,20 @@ namespace RT64 {
descSet->setTexture(descSet->TMEM, dstTexture->tmem.get(), RenderTextureLayout::SHADER_READ);
descSet->setTexture(descSet->RGBA32, dstTexture->texture.get(), RenderTextureLayout::GENERAL);
beforeDecodeBarriers.emplace_back(dstTexture->texture.get(), RenderTextureLayout::GENERAL);
// If the databases that were loaded have different hash versions, we have much to to check for all possible replacements with all possible hashes.
for (uint32_t hashVersion : textureMap.replacementMap.resolvedHashVersions) {
// If the database uses an older hash version, we hash TMEM again with the version corresponding to the database.
uint64_t databaseHash = upload.hash;
if (hashVersion < TMEMHasher::CurrentHashVersion) {
databaseHash = TMEMHasher::hash(upload.bytesTMEM.data(), upload.loadTile, upload.width, upload.height, upload.tlut, hashVersion);
}
// Add this hash so it's checked for a replacement.
replacementQueueCopy.emplace_back(ReplacementCheck{ upload.hash, databaseHash });
}
}
else {
// Hash version differences from database versions are not possible when TMEM doesn't need to be decoded.
replacementQueueCopy.emplace_back(ReplacementCheck{ upload.hash, upload.hash });
}
addReplacementChecks(upload.hash, upload.width, upload.height, upload.tlut, upload.loadTile, upload.bytesTMEM, upload.decodeTMEM, replacementQueueCopy);
}
replacementMapAdditions.clear();
for (const ReplacementCheck &replacementCheck : replacementQueueCopy) {
ReplacementResolvedPath resolvedPath;
if (textureMap.replacementMap.getResolvedPathFromHash(replacementCheck.databaseHash, resolvedPath)) {
Texture *replacementTexture = textureMap.replacementMap.getFromRelativePath(resolvedPath.relativePath);
if (textureMap.replacementMap.getResolvedPathFromHash(replacementCheck.databaseHash, replacementCheck.databaseVersion, resolvedPath)) {
Texture *replacementTexture = textureMap.replacementMap.getFromRelativePath(resolvedPath.fileSystemIndex, resolvedPath.relativePath);
Texture *lowMipCacheTexture = nullptr;
// Look for the low mip cache version if it exists if we can't use the real replacement yet.
if ((replacementTexture == nullptr) && (resolvedPath.operation == ReplacementOperation::Stream)) {
if ((replacementTexture == nullptr) && (resolvedPath.resolvedOperation == ReplacementOperation::Stream)) {
auto lowMipCacheIt = textureMap.replacementMap.lowMipCacheTextures.find(resolvedPath.relativePath);
if (lowMipCacheIt != textureMap.replacementMap.lowMipCacheTextures.end()) {
lowMipCacheTexture = lowMipCacheIt->second.texture;
@@ -1146,32 +1153,33 @@ namespace RT64 {
// Replacement texture hasn't been loaded yet.
if (replacementTexture == nullptr) {
// Queue the texture for being loaded from a texture cache streaming thread.
if (resolvedPath.operation == ReplacementOperation::Stream) {
if (resolvedPath.resolvedOperation == ReplacementOperation::Stream) {
# if !ONLY_USE_LOW_MIP_CACHE
// Make sure the replacement map hasn't queued or loaded the relative path already.
if (textureMap.replacementMap.streamRelativePathSet.find(resolvedPath.relativePath) == textureMap.replacementMap.streamRelativePathSet.end()) {
textureMap.replacementMap.streamRelativePathSet.insert(resolvedPath.relativePath);
auto &streamSet = textureMap.replacementMap.fileSystemStreamSets[resolvedPath.fileSystemIndex];
if (streamSet.find(resolvedPath.relativePath) == streamSet.end()) {
streamSet.insert(resolvedPath.relativePath);
// Push to the streaming queue.
streamDescStackMutex.lock();
streamDescStack.push(StreamDescription(resolvedPath.relativePath, false));
streamDescStack.push(StreamDescription(resolvedPath.fileSystemIndex, resolvedPath.relativePath, false));
streamDescStackMutex.unlock();
streamDescStackChanged.notify_all();
}
# endif
// Store the hash to check for the replacement when the texture is done streaming.
streamPendingReplacementChecks.emplace(resolvedPath.relativePath, replacementCheck);
textureMap.replacementMap.fileSystemStreamChecks[resolvedPath.fileSystemIndex].emplace(resolvedPath.relativePath, replacementCheck);
// Use the low mip cache texture if it exists.
replacementTexture = lowMipCacheTexture;
}
// Load the texture directly on this thread (operation was defined as Stall).
else if (textureMap.replacementMap.fileSystem->load(resolvedPath.relativePath, replacementBytes)) {
else if (textureMap.replacementMap.fileSystems[resolvedPath.fileSystemIndex]->load(resolvedPath.relativePath, replacementBytes)) {
replacementUploadResources.emplace_back();
replacementTexture = TextureCache::loadTextureFromBytes(copyWorker->device, copyWorker->commandList.get(), replacementBytes, replacementUploadResources.back());
textureMapMutex.lock();
textureMap.replacementMap.addLoadedTexture(replacementTexture, resolvedPath.relativePath, true);
textureMap.replacementMap.addLoadedTexture(replacementTexture, resolvedPath.fileSystemIndex, resolvedPath.relativePath, true);
textureMapMutex.unlock();
afterDecodeBarriers.emplace_back(replacementTexture->texture.get(), RenderTextureLayout::SHADER_READ);
}
@@ -1179,8 +1187,8 @@ namespace RT64 {
if (replacementTexture != nullptr) {
// We don't use reference counting on preloaded textures or low mip cache versions, as they're permanently allocated in memory.
bool referenceCounted = (resolvedPath.operation != ReplacementOperation::Preload) && (replacementTexture != lowMipCacheTexture);
replacementMapAdditions.emplace_back(ReplacementMapAddition{ replacementCheck.textureHash, replacementTexture, referenceCounted });
bool referenceCounted = (resolvedPath.resolvedOperation != ReplacementOperation::Preload) && (replacementTexture != lowMipCacheTexture);
replacementMapAdditions.emplace_back(ReplacementMapAddition{ replacementCheck.textureHash, replacementTexture, resolvedPath.resolvedShift, referenceCounted });
}
}
}
@@ -1248,7 +1256,7 @@ namespace RT64 {
}
for (const ReplacementMapAddition &addition : replacementMapAdditions) {
textureMap.replace(addition.hash, addition.texture, addition.referenceCounted);
textureMap.replace(addition.hash, addition.texture, addition.shift == ReplacementShift::Half, addition.referenceCounted);
}
textureMap.replacementMap.evict(textureMap.evictedTextures);
@@ -1295,9 +1303,33 @@ namespace RT64 {
});
}
bool TextureCache::useTexture(uint64_t hash, uint64_t submissionFrame, uint32_t &textureIndex, interop::float2 &textureScale, interop::float3 &textureDimensions, bool &textureReplaced, bool &hasMipmaps) {
void TextureCache::addReplacementChecks(uint64_t hash, uint32_t width, uint32_t height, uint32_t tlut, const LoadTile &loadTile, const std::vector<uint8_t> &bytesTMEM, bool decodeTMEM, std::vector<ReplacementCheck> &replacementChecks, uint64_t exclusiveDbHash) {
if (decodeTMEM) {
// If the databases that were loaded have different hash versions, we have much to to check for all possible replacements with all possible hashes.
for (uint32_t hashVersion : textureMap.replacementMap.resolvedHashVersions) {
// If the database uses an older hash version, we hash TMEM again with the version corresponding to the database.
uint64_t databaseHash = hash;
if (hashVersion < TMEMHasher::CurrentHashVersion) {
databaseHash = TMEMHasher::hash(bytesTMEM.data(), loadTile, width, height, tlut, hashVersion);
}
if ((exclusiveDbHash == 0) || (exclusiveDbHash == databaseHash)) {
// Add this hash so it's checked for a replacement.
replacementChecks.emplace_back(ReplacementCheck{ hash, databaseHash, hashVersion });
}
}
}
else {
if ((exclusiveDbHash == 0) || (exclusiveDbHash == hash)) {
// Hash version differences from database versions are not possible when TMEM doesn't need to be decoded.
replacementChecks.emplace_back(ReplacementCheck{ hash, hash, TMEMHasher::CurrentHashVersion });
}
}
}
bool TextureCache::useTexture(uint64_t hash, uint64_t submissionFrame, uint32_t &textureIndex, interop::float2 &textureScale, interop::float3 &textureDimensions, bool &textureReplaced, bool &hasMipmaps, bool &shiftedByHalf) {
std::unique_lock lock(textureMapMutex);
return textureMap.use(hash, submissionFrame, textureIndex, textureScale, textureDimensions, textureReplaced, hasMipmaps);
return textureMap.use(hash, submissionFrame, textureIndex, textureScale, textureDimensions, textureReplaced, hasMipmaps, shiftedByHalf);
}
bool TextureCache::useTexture(uint64_t hash, uint64_t submissionFrame, uint32_t &textureIndex) {
@@ -1305,23 +1337,28 @@ namespace RT64 {
interop::float3 textureDimensions;
bool textureReplaced;
bool hasMipmaps;
return useTexture(hash, submissionFrame, textureIndex, textureScale, textureDimensions, textureReplaced, hasMipmaps);
bool shiftedByHalf;
return useTexture(hash, submissionFrame, textureIndex, textureScale, textureDimensions, textureReplaced, hasMipmaps, shiftedByHalf);
}
bool TextureCache::addReplacement(uint64_t hash, const std::string &relativePath) {
bool TextureCache::addReplacement(uint64_t hash, const std::string &relativePath, ReplacementShift shift) {
waitForAllStreamThreads(false);
waitForGPUUploads();
std::unique_lock lock(textureMapMutex);
if (!textureMap.replacementMap.fileSystemIsDirectory) {
return false;
}
std::vector<uint8_t> replacementBytes;
if (!textureMap.replacementMap.fileSystem->load(relativePath, replacementBytes)) {
if (!textureMap.replacementMap.fileSystems.front()->load(relativePath, replacementBytes)) {
return false;
}
// Load texture replacement immediately.
bool loadedNewTexture = false;
std::string relativePathForward = FileSystem::toForwardSlashes(relativePath);
Texture *newTexture = textureMap.replacementMap.getFromRelativePath(relativePathForward);
Texture *newTexture = textureMap.replacementMap.getFromRelativePath(0, relativePathForward);
if (newTexture == nullptr) {
std::unique_ptr<RenderBuffer> dstUploadBuffer;
loaderCommandList->begin();
@@ -1344,21 +1381,25 @@ namespace RT64 {
ReplacementTexture replacement;
replacement.hashes.rt64 = ReplacementDatabase::hashToString(hash);
replacement.path = ReplacementDatabase::removeKnownExtension(relativePathForward);
replacement.shift = shift;
// Add the replacement's index to the resolved path map as well.
uint32_t databaseIndex = textureMap.replacementMap.directoryDatabase.addReplacement(replacement);
textureMap.replacementMap.resolvedPathMap[hash] = { relativePathForward, ReplacementOperation::Stream };
ReplacementDatabase &replacementDb = textureMap.replacementMap.directoryDatabase;
replacementDb.addReplacement(replacement);
ReplacementOperation resolvedOperation = replacementDb.resolveOperation(replacement.path, replacement.operation);
ReplacementShift resolvedShift = replacementDb.resolveShift(replacement.path, replacement.shift);
textureMap.replacementMap.resolvedPathMaps[TMEMHasher::CurrentHashVersion][hash] = ReplacementResolvedPath{ 0, relativePathForward, resolvedOperation, replacement.operation, resolvedShift, replacement.shift };
uploadQueueMutex.lock();
// Queue the texture as if it was the result from a streaming thread.
if (loadedNewTexture) {
streamResultQueue.emplace_back(newTexture, relativePathForward, false);
streamPendingReplacementChecks.emplace(relativePathForward, ReplacementCheck{ hash, hash });
streamResultQueue.emplace_back(newTexture, 0, relativePathForward, false);
textureMap.replacementMap.fileSystemStreamChecks[0].emplace(relativePathForward, ReplacementCheck{hash, hash, TMEMHasher::CurrentHashVersion});
}
// Just queue a replacement check for the hash that was just replaced.
else {
replacementQueue.emplace_back(ReplacementCheck{ hash, hash });
replacementQueue.emplace_back(ReplacementCheck{ hash, hash, TMEMHasher::CurrentHashVersion });
}
uploadQueueMutex.unlock();
@@ -1369,7 +1410,13 @@ namespace RT64 {
bool TextureCache::hasReplacement(uint64_t hash) {
std::unique_lock lock(textureMapMutex);
return (textureMap.replacementMap.resolvedPathMap.find(hash) != textureMap.replacementMap.resolvedPathMap.end());
for (uint32_t version : textureMap.replacementMap.resolvedHashVersions) {
if (textureMap.replacementMap.resolvedPathMaps[version].find(hash) != textureMap.replacementMap.resolvedPathMaps[version].end()) {
return true;
}
}
return false;
}
void TextureCache::clearReplacementDirectories() {
@@ -1390,10 +1437,10 @@ namespace RT64 {
}
// Clear the current set of textures that were sent to streaming threads.
textureMap.replacementMap.streamRelativePathSet.clear();
textureMap.replacementMap.fileSystemStreamSets.clear();
// Clear the pending replacement checks for streamed textures.
streamPendingReplacementChecks.clear();
textureMap.replacementMap.fileSystemStreamChecks.clear();
// Lock the texture map and start changing replacements. This function is assumed to be called from the only
// thread that is capable of submitting new textures and must've waited beforehand for all textures to be uploaded.
@@ -1402,6 +1449,25 @@ namespace RT64 {
textureMap.replacementMap.clear(textureMap.evictedTextures);
}
void TextureCache::reloadReplacements(bool clearQueue, uint64_t exclusiveDbHash) {
// Queue all currently loaded hashes to detect replacements with.
{
std::unique_lock queueLock(uploadQueueMutex);
if (clearQueue) {
replacementQueue.clear();
}
for (size_t i = 0; i < textureMap.textures.size(); i++) {
if (textureMap.textures[i] != nullptr) {
Texture *texture = textureMap.textures[i];
addReplacementChecks(textureMap.hashes[i], texture->width, texture->height, texture->tlut, texture->loadTile, texture->bytesTMEM, texture->decodeTMEM, replacementQueue, exclusiveDbHash);
}
}
}
uploadQueueChanged.notify_all();
}
bool TextureCache::loadReplacementDirectory(const ReplacementDirectory &replacementDirectory) {
return loadReplacementDirectories({ replacementDirectory });
}
@@ -1413,6 +1479,7 @@ namespace RT64 {
textureMap.replacementMap.replacementDirectories = replacementDirectories;
std::vector<std::unique_ptr<FileSystem>> fileSystems;
std::vector<std::unordered_set<std::string>> fileSystemStreamSets;
for (const ReplacementDirectory &replacementDirectory : replacementDirectories) {
if (std::filesystem::is_regular_file(replacementDirectory.dirOrZipPath)) {
std::unique_ptr<FileSystem> fileSystem = FileSystemZip::create(replacementDirectory.dirOrZipPath, replacementDirectory.zipBasePath);
@@ -1422,6 +1489,7 @@ namespace RT64 {
}
fileSystems.emplace_back(std::move(fileSystem));
fileSystemStreamSets.emplace_back();
}
else if (std::filesystem::is_directory(replacementDirectory.dirOrZipPath)) {
std::unique_ptr<FileSystem> fileSystem = FileSystemDirectory::create(replacementDirectory.dirOrZipPath);
@@ -1433,6 +1501,7 @@ namespace RT64 {
// Only enable that file system is a directory if it's only one directory.
textureMap.replacementMap.fileSystemIsDirectory = (replacementDirectories.size() == 1);
fileSystems.emplace_back(std::move(fileSystem));
fileSystemStreamSets.emplace_back();
}
else {
fprintf(stderr, "Failed to identify what type of filesystem the path is.\n");
@@ -1440,8 +1509,7 @@ namespace RT64 {
}
}
// Load databases and low mipmap caches from the filesystems in reverse order.
std::unordered_set<uint64_t> hashesToPreload;
// Load databases and low mipmap caches from the filesystems in reverse order.
{
loaderCommandList->begin();
@@ -1458,7 +1526,7 @@ namespace RT64 {
db = json::parse(databaseBytes.begin(), databaseBytes.end(), nullptr, true);
if (db.config.hashVersion <= TMEMHasher::CurrentHashVersion) {
db.resolvePaths(fileSystems[i].get(), textureMap.replacementMap.resolvedPathMap, false, nullptr, &hashesToPreload);
db.resolvePaths(fileSystems[i].get(), uint32_t(i), textureMap.replacementMap.resolvedPathMaps[db.config.hashVersion], false, nullptr, &fileSystemStreamSets[i]);
if (!knownHashVersions[db.config.hashVersion]) {
textureMap.replacementMap.resolvedHashVersions.insert(textureMap.replacementMap.resolvedHashVersions.begin(), db.config.hashVersion);
@@ -1495,49 +1563,32 @@ namespace RT64 {
}
}
// Create a combined file system out of the systems that were loaded.
if (fileSystems.size() > 1) {
textureMap.replacementMap.fileSystem = FileSystemCombined::create(fileSystems);
}
else {
textureMap.replacementMap.fileSystem = std::move(fileSystems.front());
}
// Store the file systems in the replacement map.
uint32_t fileSystemCount = uint32_t(fileSystems.size());
textureMap.replacementMap.fileSystems = std::move(fileSystems);
textureMap.replacementMap.fileSystemStreamSets = std::move(fileSystemStreamSets);
textureMap.replacementMap.fileSystemStreamChecks.resize(fileSystemCount);
const bool preloadTexturesWithThreads = !hashesToPreload.empty();
if (preloadTexturesWithThreads) {
// Queue all textures that must be preloaded to the stream queues.
{
std::unique_lock queueLock(streamDescStackMutex);
for (uint64_t hash : hashesToPreload) {
auto it = textureMap.replacementMap.resolvedPathMap.find(hash);
const std::string &relativePath = it->second.relativePath;
if (textureMap.replacementMap.streamRelativePathSet.find(relativePath) == textureMap.replacementMap.streamRelativePathSet.end()) {
streamDescStack.push(StreamDescription(relativePath, true));
textureMap.replacementMap.streamRelativePathSet.insert(relativePath);
}
// Queue all textures that must be preloaded to the stream queues.
bool texturesPreloaded = false;
{
std::unique_lock queueLock(streamDescStackMutex);
for (uint32_t i = 0; i < fileSystemCount; i++) {
for (const std::string &relativePath : textureMap.replacementMap.fileSystemStreamSets[i]) {
streamDescStack.push(StreamDescription(i, relativePath, true));
texturesPreloaded = true;
}
}
streamDescStackChanged.notify_all();
}
if (preloadTexturesWithThreads) {
if (texturesPreloaded) {
streamDescStackChanged.notify_all();
// Wait for all the streaming threads to be finished.
waitForAllStreamThreads(false);
}
// Queue all currently loaded hashes to detect replacements with.
{
std::unique_lock queueLock(uploadQueueMutex);
replacementQueue.clear();
for (size_t i = 0; i < textureMap.hashes.size(); i++) {
if (textureMap.hashes[i] != 0) {
replacementQueue.emplace_back(ReplacementCheck{ textureMap.hashes[i], textureMap.hashes[i] });
}
}
}
uploadQueueChanged.notify_all();
reloadReplacements(true);
return true;
}
@@ -1585,6 +1636,58 @@ namespace RT64 {
return true;
}
void TextureCache::setReplacementDefaultShift(ReplacementShift shift) {
std::unique_lock lock(textureMapMutex);
ReplacementDatabase &replacementDb = textureMap.replacementMap.directoryDatabase;
replacementDb.config.defaultShift = shift;
replacementDb.resolveShifts(textureMap.replacementMap.resolvedPathMaps[replacementDb.config.hashVersion]);
reloadReplacements(false);
}
void TextureCache::setReplacementDefaultOperation(ReplacementOperation operation) {
std::unique_lock lock(textureMapMutex);
ReplacementDatabase &replacementDb = textureMap.replacementMap.directoryDatabase;
replacementDb.config.defaultOperation = operation;
replacementDb.resolveOperations(textureMap.replacementMap.resolvedPathMaps[replacementDb.config.hashVersion]);
reloadReplacements(false);
}
void setReplacementTextureOperationOrShift(TextureCache &cache, uint64_t hash, ReplacementTexture texture) {
ReplacementDatabase &replacementDb = cache.textureMap.replacementMap.directoryDatabase;
replacementDb.addReplacement(texture);
auto it = cache.textureMap.replacementMap.resolvedPathMaps[replacementDb.config.hashVersion].find(hash);
if (it != cache.textureMap.replacementMap.resolvedPathMaps[replacementDb.config.hashVersion].end()) {
it->second.originalOperation = texture.operation;
it->second.originalShift = texture.shift;
it->second.resolvedOperation = replacementDb.resolveOperation(it->second.relativePath, texture.operation);
it->second.resolvedShift = replacementDb.resolveShift(it->second.relativePath, texture.shift);
// Since we don't have exact mappings from the database hashes to the cache hashes, we run some hash checks
// on all currently loaded textures to check for replacements.
cache.reloadReplacements(false, hash);
}
}
void TextureCache::setReplacementShift(uint64_t hash, ReplacementShift shift) {
std::unique_lock lock(textureMapMutex);
ReplacementDatabase &replacementDb = textureMap.replacementMap.directoryDatabase;
ReplacementTexture replacement = replacementDb.getReplacement(hash);
if (!replacement.isEmpty()) {
replacement.shift = shift;
setReplacementTextureOperationOrShift(*this, hash, replacement);
}
}
void TextureCache::setReplacementOperation(uint64_t hash, ReplacementOperation operation) {
std::unique_lock lock(textureMapMutex);
ReplacementDatabase &replacementDb = textureMap.replacementMap.directoryDatabase;
ReplacementTexture replacement = replacementDb.getReplacement(hash);
if (!replacement.isEmpty()) {
replacement.operation = operation;
setReplacementTextureOperationOrShift(*this, hash, replacement);
}
}
void TextureCache::removeUnusedEntriesFromDatabase() {
std::unique_lock lock(textureMapMutex);
if (!textureMap.replacementMap.fileSystemIsDirectory) {

54
src/render/rt64_texture_cache.h
View File

@@ -64,8 +64,15 @@ namespace RT64 {
}
};
struct ReplacementCheck {
uint64_t textureHash = 0;
uint64_t databaseHash = 0;
uint32_t databaseVersion = 0;
};
struct ReplacementMap {
struct MapEntry {
uint32_t fileSystemIndex = 0;
std::string relativePath;
Texture *texture = nullptr;
uint32_t references = 0;
@@ -78,11 +85,12 @@ namespace RT64 {
LoadedTextureMap loadedTextureMap;
LoadedTextureReverseMap loadedTextureReverseMap;
std::list<Texture *> unusedTextureList;
std::unordered_set<std::string> streamRelativePathSet;
std::unordered_map<uint64_t, ReplacementResolvedPath> resolvedPathMap;
std::unordered_map<uint64_t, ReplacementResolvedPath> resolvedPathMaps[6];
std::vector<uint32_t> resolvedHashVersions;
std::unordered_map<std::string, LowMipCacheTexture> lowMipCacheTextures;
std::unique_ptr<FileSystem> fileSystem;
std::vector<std::unique_ptr<FileSystem>> fileSystems;
std::vector<std::unordered_set<std::string>> fileSystemStreamSets;
std::vector<std::unordered_multimap<std::string, ReplacementCheck>> fileSystemStreamChecks;
std::vector<ReplacementDirectory> replacementDirectories;
uint64_t usedTexturePoolSize = 0;
uint64_t cachedTexturePoolSize = 0;
@@ -96,19 +104,14 @@ namespace RT64 {
void evict(std::vector<Texture *> &evictedTextures);
bool saveDatabase(std::ostream &stream);
void removeUnusedEntriesFromDatabase();
bool getResolvedPathFromHash(uint64_t tmemHash, ReplacementResolvedPath &resolvedPath) const;
void addLoadedTexture(Texture *texture, const std::string &relativePath, bool referenceCounted);
Texture *getFromRelativePath(const std::string &relativePath) const;
uint64_t hashFromRelativePath(const std::string &relativePath) const;
bool getResolvedPathFromHash(uint64_t dbHash, uint32_t dbHashVersion, ReplacementResolvedPath &resolvedPath) const;
void addLoadedTexture(Texture *texture, uint32_t fileSystemIndex, const std::string &relativePath, bool referenceCounted);
Texture *getFromRelativePath(uint32_t fileSystemIndex, const std::string &relativePath) const;
uint64_t hashFromRelativePath(uint32_t fileSystemIndex, const std::string &relativePath) const;
void incrementReference(Texture *texture);
void decrementReference(Texture *texture);
};
struct ReplacementCheck {
uint64_t textureHash = 0;
uint64_t databaseHash = 0;
};
struct TextureMapAddition {
uint64_t hash = 0;
Texture *texture = nullptr;
@@ -117,6 +120,7 @@ namespace RT64 {
struct ReplacementMapAddition {
uint64_t hash = 0;
Texture *texture = nullptr;
ReplacementShift shift = ReplacementShift::None;
bool referenceCounted = false;
};
@@ -126,6 +130,7 @@ namespace RT64 {
std::vector<interop::float3> cachedTextureDimensions;
std::vector<Texture *> textureReplacements;
std::vector<interop::float3> cachedTextureReplacementDimensions;
std::vector<bool> textureReplacementShiftedByHalf;
std::vector<bool> textureReplacementReferenceCounted;
std::vector<interop::float2> textureScales;
std::vector<uint64_t> hashes;
@@ -143,8 +148,8 @@ namespace RT64 {
~TextureMap();
void clearReplacements();
void add(uint64_t hash, uint64_t creationFrame, Texture *texture);
void replace(uint64_t hash, Texture *texture, bool referenceCounted);
bool use(uint64_t hash, uint64_t submissionFrame, uint32_t &textureIndex, interop::float2 &textureScale, interop::float3 &textureDimensions, bool &textureReplaced, bool &hasMipmaps);
void replace(uint64_t hash, Texture *texture, bool shiftedByHalf, bool referenceCounted);
bool use(uint64_t hash, uint64_t submissionFrame, uint32_t &textureIndex, interop::float2 &textureScale, interop::float3 &textureDimensions, bool &textureReplaced, bool &hasMipmaps, bool &shiftedByHalf);
bool evict(uint64_t submissionFrame, std::vector<uint64_t> &evictedHashes);
void incrementLock();
void decrementLock();
@@ -154,6 +159,7 @@ namespace RT64 {
struct TextureCache {
struct StreamDescription {
uint32_t fileSystemIndex = 0;
std::string relativePath;
bool fromPreload = false;
@@ -161,7 +167,8 @@ namespace RT64 {
// Default constructor.
}
StreamDescription(const std::string &relativePath, bool fromPreload) {
StreamDescription(uint32_t fileSystemIndex, const std::string &relativePath, bool fromPreload) {
this->fileSystemIndex = fileSystemIndex;
this->relativePath = relativePath;
this->fromPreload = fromPreload;
}
@@ -169,6 +176,7 @@ namespace RT64 {
struct StreamResult {
Texture *texture = nullptr;
uint32_t fileSystemIndex = 0;
std::string relativePath;
bool fromPreload = false;
@@ -176,8 +184,9 @@ namespace RT64 {
// Default constructor.
}
StreamResult(Texture *texture, const std::string &relativePath, bool fromPreload) {
StreamResult(Texture *texture, uint32_t fileSystemIndex, const std::string &relativePath, bool fromPreload) {
this->texture = texture;
this->fileSystemIndex = fileSystemIndex;
this->relativePath = relativePath;
this->fromPreload = fromPreload;
}
@@ -212,7 +221,6 @@ namespace RT64 {
std::condition_variable streamDescStackChanged;
int32_t streamDescStackActiveCount = 0;
std::list<std::unique_ptr<StreamThread>> streamThreads;
std::unordered_multimap<std::string, ReplacementCheck> streamPendingReplacementChecks;
std::mutex streamPerformanceMutex;
uint64_t streamLoadTimeTotal = 0;
uint64_t streamLoadCount = 0;
@@ -228,19 +236,25 @@ namespace RT64 {
uint32_t lockCounter;
bool developerMode;
TextureCache(RenderWorker *directWorker, RenderWorker *copyWorker, uint32_t threadCount, const ShaderLibrary *shaderLibrary, bool developerMode);
TextureCache(RenderWorker *directWorker, RenderWorker *copyWorker, uint32_t threadCount, const ShaderLibrary *shaderLibrary);
~TextureCache();
void uploadThreadLoop();
void queueGPUUploadTMEM(uint64_t hash, uint64_t creationFrame, const uint8_t *bytes, int bytesCount, int width, int height, uint32_t tlut, const LoadTile &loadTile, bool decodeTMEM);
void waitForGPUUploads();
bool useTexture(uint64_t hash, uint64_t submissionFrame, uint32_t &textureIndex, interop::float2 &textureScale, interop::float3 &textureDimensions, bool &textureReplaced, bool &hasMipmaps);
void addReplacementChecks(uint64_t hash, uint32_t width, uint32_t height, uint32_t tlut, const LoadTile &loadTile, const std::vector<uint8_t> &bytesTMEM, bool decodeTMEM, std::vector<ReplacementCheck> &replacementChecks, uint64_t exclusiveDbHash = 0);
bool useTexture(uint64_t hash, uint64_t submissionFrame, uint32_t &textureIndex, interop::float2 &textureScale, interop::float3 &textureDimensions, bool &textureReplaced, bool &hasMipmaps, bool &shiftedByHalf);
bool useTexture(uint64_t hash, uint64_t submissionFrame, uint32_t &textureIndex);
bool addReplacement(uint64_t hash, const std::string &relativePath);
bool addReplacement(uint64_t hash, const std::string &relativePath, ReplacementShift shift);
bool hasReplacement(uint64_t hash);
void clearReplacementDirectories();
void reloadReplacements(bool clearQueue, uint64_t exclusiveDbHash = 0);
bool loadReplacementDirectory(const ReplacementDirectory &replacementDirectory);
bool loadReplacementDirectories(const std::vector<ReplacementDirectory> &replacementDirectories);
bool saveReplacementDatabase();
void setReplacementDefaultShift(ReplacementShift shift);
void setReplacementDefaultOperation(ReplacementOperation shift);
void setReplacementShift(uint64_t hash, ReplacementShift shift);
void setReplacementOperation(uint64_t hash, ReplacementOperation operation);
void removeUnusedEntriesFromDatabase();
bool evict(uint64_t submissionFrame, std::vector<uint64_t> &evictedHashes);
void incrementLock();

5
src/shaders/TextureSampler.hlsli
View File

@@ -235,6 +235,11 @@ float4 sampleTexture(OtherMode otherMode, RenderFlags renderFlags, float2 inputU
uvCoord.x += 1.0f;
}
const bool flagShiftedByHalf = gpuTileFlagShiftedByHalf(gpuTile.flags);
if (flagShiftedByHalf) {
uvCoord += float2(0.5f, 0.5f);
}
uvCoord *= gpuTile.tcScale;
uvCoord -= (float2(rdpTile.uls, rdpTile.ult) * gpuTile.ulScale) / 4.0f;

5
src/shared/rt64_gpu_tile.h
View File

@@ -17,6 +17,7 @@ namespace interop {
uint fromCopy : 1;
uint rawTMEM : 1;
uint hasMipmaps : 1;
uint shiftedByHalf : 1;
};
uint value;
@@ -44,6 +45,10 @@ namespace interop {
bool gpuTileFlagHasMipmaps(GPUTileFlags flags) {
return flags & 0x10;
}
bool gpuTileFlagShiftedByHalf(GPUTileFlags flags) {
return flags & 0x20;
}
#endif
struct GPUTile {
float2 ulScale;

4
src/tools/texture_packer/texture_packer.cpp
View File

@@ -342,10 +342,10 @@ int main(int argc, char *argv[]) {
std::vector<uint64_t> hashesMissing;
std::unordered_map<uint64_t, RT64::ReplacementResolvedPath> resolvedPathMap;
std::unique_ptr<RT64::FileSystem> fileSystem = RT64::FileSystemDirectory::create(searchDirectory);
database.resolvePaths(fileSystem.get(), resolvedPathMap, mode == Mode::CreateLowMipCache, &hashesMissing);
database.resolvePaths(fileSystem.get(), 0, resolvedPathMap, mode == Mode::CreateLowMipCache, &hashesMissing);
for (auto it : resolvedPathMap) {
if ((mode != Mode::CreateLowMipCache) || (it.second.operation == RT64::ReplacementOperation::Stream)) {
if ((mode != Mode::CreateLowMipCache) || (it.second.resolvedOperation == RT64::ReplacementOperation::Stream)) {
resolvedPathSet.insert(it.second.relativePath);
}
}