Difference between revisions of "Paired ARC+DIR file"

From Pikmin Technical Knowledge Base
Jump to navigation Jump to search
(I wrote a bad article. Will fix over time.)
 
m
 
(14 intermediate revisions by 4 users not shown)
Line 1: Line 1:
−
Pikmin 1 has a very stupid ARC format that isn't compatible with LunaBoy's RARC tools and makes everyone very sad.
+
'''ARC''' files (short for '''Archive''') and '''DIR''' files (internally named '''Bundle''' ; not to be confused with [[BUN file|'''BUN''' files]]) make up one of ''Pikmin''<nowiki>'s</nowiki> proprietary data storage formats. The '''DIR''' file contains all necessary information about the data in the '''ARC''' file, which stores the aforementioned data.
  
−
However, that doesn't mean that it is difficult to understand-- quite the contrary.
+
With minimal metadata, the Bundled Archive file format was used to reduce load time by packing several files into one. The purpose it serves is different from the [[BUN file|bundle]] archive, however, as standalone files within the archive can be read independently of the files before it.  This makes the Bundled Archive file format useful for archiving large directories with varied purposes, as to its name.
  
 
__TOC__
 
__TOC__
  
−
==The Basics==
+
== File structure ==
−
A lot of this is currently theory and unprofessional sounding, but hear me out.
+
As mentioned before, the paired ARC+DIR file archive consists of two files: the mock-directory (DIR) and the BLOB archive (ARC).  Think of the mock-directory as a map to navigate through the BLOB archive.
−
The developers of Pikmin, unsure of how big of a game they were going to make, made a very simple compression script for their files.  This script would take a directory or subdirectory in <code>root\datadir</code> and create a compressed copy of all the files in <code>root\datadir\archives</code>.  This is the ARC file, it is named after the lowest directory accessed, and simply put, it is every file in that directory smushed together with loads of <code>0xCCCC</code> between them to differentiate.
 
  
−
How many sets of <code>0xCCCC</code>?  I don't know yet.
+
=== DIR ===
 +
DIR files consist of a header followed by a repeating entry for each mock-filepath.
  
−
Anyway, this isn't all their script would do. It would also create a partner '''DIR''' file describing the directory path, treating datadir as the root, to each file that was stored in the ARC.  It's sort of like emulating the directory path I think, plus it tells us the order of the files in the ARC file.
+
Here is <code>/dataDir/archives/pelletsbin.dir</code> annotated for reference.
  
−
Oh and also the four bytes starting at 0x4 is the size of the DIR file in bytes. Just thought I'd share that.
+
[[File:DIR_file_documentation.png]][[File:Minty_Meeo_Documentation_Legend.png]]
  
 +
{| class="wikitable"
 +
|+ Header
 +
! Offset || Size || Purpose
 +
|-
 +
| 0x00 || 4 || Size of DIR file
 +
|-
 +
| 0x04 || 4 || Number of entries
 +
|-
 +
|}
  
−
==The bloody hell are you getting on about?==
+
{| class="wikitable"  
−
So I have been trying to get Beady Long Legs to leave its arena on console.  I specify "on console" because I have been able to get him to leave his arena in the Windows x86 debugger for 6 1/2 months, but no matter what I did, the edits I did to the files on PC didn't change anything on console.  Well, today I was finally able to get BLL OoA on console/Dolphin by editing the archived version of bosses\kumo\parms.bin that is stored within archives\bosses.arc.  Using bosses.dir as a guide, I was able to locate the archived copy of the parameters file, edit it, and run it on console/Dolphin.  [https://www.youtube.com/watch?v=iXJFQKinoBM You can watch it here.]  The next step is to take what I've described and make a tool, right?  I don't know any good programming languages, and if I'm the one making the tool, get ready to wait another 6 1/2 months.
+
|+ Mock-directory entry
 +
! Offset || Size || Purpose
 +
|-
 +
| 0x00 || 4 || Offset of file in ARC
 +
|-
 +
| 0x04 || 4 || Size of file in ARC
 +
|-
 +
| 0x08 || 4 || Mock-directory string length
 +
|-
 +
| 0x0C || 0x08 value || Mock-directory path
 +
|-
 +
|}
  
 +
==== Mock-directory ====
 +
The DIR file emulates filepaths.  When a file is unpacked in-game, it is assumed that whatever code is responsible treats the entry in the DIR file as a legitimate filepath on the disc, starting from the root.
  
−
==Summary==
+
=== ARC ===
−
Why haven't we figured out this format yet we have carbon copies of every file in every ARC present on the disc.  Someone make an unpacker+packer so we can hack this gaem pls and thank.
+
Here is <code>/dataDir/archives/pelletsbin.arc</code> annotated to show how mapping the ARC file would look like with information from the corresponding DIR. Without the DIR file as a guide, the only way you could possibly differentiate between files would be with the <code>0xCC</code> padding between files.
 +
 
 +
[[File:ARC_file_documentation.png]][[File:Minty_Meeo_Documentation_Legend.png]]
 +
 
 +
==== <code>0xCC</code> padding ====
 +
Files in the ARC always have padding to nearest multiple of 32, or 0x20. This is because data loading into the Gamecube's register's in multiples of 32 is an optimization.
 +
 
 +
== Extra information ==
 +
Every paired Bundled Archive file archive has an uncompressed copy elsewhere on the disc.
 +
* In <code>/dataDir/tekis</code> and <code>/dataDir/bosses</code>, a file named <code>arc_all.ini</code> can be found.  These INI files holds nothing, but are an indicator to the archiving tool to bundle the directory's contents.
 +
* It is unknown if every file in the archives is identical to its uncompressed sister file.
 +
In the [[OpenGL / Dolphin System]], many files are not read from the paired ARC+DIR file archives like they are in the final game, and are instead read from the un-archived versions. 
 +
 
 +
Bundled Archives that have "bin" in their filename only contain the '''BIN''' files from their respective directory.
 +
 
 +
The <code>log.txt</code> files left in the root of [[Pikmin|''Pikmin'' versions USA Demo, USA v1.00, and JPN v1.02]] detail the plugin "[[OpenGL / Dolphin System|plugTexConv]]" creating a paired ARC+DIR file archive.
 +
 
 +
{{Credits|[[User:Minty_Meeo|Minty_Meeo]], [[User:Patrick_Anton|Ambrosia]]}}
  
 
[[Category:Pikmin]]
 
[[Category:Pikmin]]
 
[[Category:File formats]]
 
[[Category:File formats]]

Latest revision as of 17:35, 29 December 2019

ARC files (short for Archive) and DIR files (internally named Bundle ; not to be confused with BUN files) make up one of Pikmin's proprietary data storage formats. The DIR file contains all necessary information about the data in the ARC file, which stores the aforementioned data.

With minimal metadata, the Bundled Archive file format was used to reduce load time by packing several files into one. The purpose it serves is different from the bundle archive, however, as standalone files within the archive can be read independently of the files before it. This makes the Bundled Archive file format useful for archiving large directories with varied purposes, as to its name.

File structure[edit]

As mentioned before, the paired ARC+DIR file archive consists of two files: the mock-directory (DIR) and the BLOB archive (ARC). Think of the mock-directory as a map to navigate through the BLOB archive.

DIR[edit]

DIR files consist of a header followed by a repeating entry for each mock-filepath.

Here is /dataDir/archives/pelletsbin.dir annotated for reference.

DIR file documentation.pngMinty Meeo Documentation Legend.png

Header
Offset Size Purpose
0x00 4 Size of DIR file
0x04 4 Number of entries
Mock-directory entry
Offset Size Purpose
0x00 4 Offset of file in ARC
0x04 4 Size of file in ARC
0x08 4 Mock-directory string length
0x0C 0x08 value Mock-directory path

Mock-directory[edit]

The DIR file emulates filepaths. When a file is unpacked in-game, it is assumed that whatever code is responsible treats the entry in the DIR file as a legitimate filepath on the disc, starting from the root.

ARC[edit]

Here is /dataDir/archives/pelletsbin.arc annotated to show how mapping the ARC file would look like with information from the corresponding DIR. Without the DIR file as a guide, the only way you could possibly differentiate between files would be with the 0xCC padding between files.

ARC file documentation.pngMinty Meeo Documentation Legend.png

0xCC padding[edit]

Files in the ARC always have padding to nearest multiple of 32, or 0x20. This is because data loading into the Gamecube's register's in multiples of 32 is an optimization.

Extra information[edit]

Every paired Bundled Archive file archive has an uncompressed copy elsewhere on the disc.

  • In /dataDir/tekis and /dataDir/bosses, a file named arc_all.ini can be found. These INI files holds nothing, but are an indicator to the archiving tool to bundle the directory's contents.
  • It is unknown if every file in the archives is identical to its uncompressed sister file.

In the OpenGL / Dolphin System, many files are not read from the paired ARC+DIR file archives like they are in the final game, and are instead read from the un-archived versions.

Bundled Archives that have "bin" in their filename only contain the BIN files from their respective directory.

The log.txt files left in the root of Pikmin versions USA Demo, USA v1.00, and JPN v1.02 detail the plugin "plugTexConv" creating a paired ARC+DIR file archive.

Credits: Minty_Meeo, Ambrosia
Retrieved from "https://pikmintkb.com/w/index.php?title=Paired_ARC%2BDIR_file&oldid=2858"