Difference between revisions of "Paired ARC+DIR file"

From Pikmin Technical Knowledge Base
Jump to navigation Jump to search
m
m (Rewrite slightly)
Line 1: Line 1:
'''ARC''' files (likely short for '''Archive''') and '''DIR''' files (technically short for '''Bundle''', though '''Directory''' makes more sense and avoids conflict with [[BUN file|'''BUN''' files]]) are two peas in a pod, together representing one of the many Binary Large Object (BLOB) archive formats used in ''[[Pikmin]]''.  Together, they will henceforth be referred to as the ''paired ARC+DIR file'' archive.
+
'''ARC''' files (short for '''Archive''') and '''DIR''' files (internally named '''Bundle''' as to not confused with [[BUN file|'''BUN''' files]]) make up one of the games proprietary data storage formats. The '''DIR''' file contains all necessary information about the data in the '''ARC''' file, which stores the aforementioned data.
  
With minimal metadata, {{unsure|the paired ARC+DIR file archive was likely used to reduce load time by packing several files into one}}. The purpose it serves is different from the [[BUN file|bundle]] archive, however, {{unsure|as standalone files within the archive can be unpacked independently of the files before it}}.  This makes the paired ARC+DIR file useful for archiving large directories with varied purposes.
+
With minimal metadata, the Bundled Archive file format was used to reduce load time by packing several files into one. The purpose it serves is different from the [[BUN file|bundle]] archive, however, as standalone files within the archive can be read independently of the files before it.  This makes the Bundled Archive file format useful for archiving large directories with varied purposes, as to it's name.
  
 
__TOC__
 
__TOC__
Line 9: Line 9:
  
 
=== DIR ===
 
=== DIR ===
DIR files consist of a header followed by a repeating struct for each mock-filepath.
+
DIR files consist of a header followed by a repeating entry for each mock-filepath.
  
 
Here is <code>/dataDir/archives/pelletsbin.dir</code> annotated for reference.
 
Here is <code>/dataDir/archives/pelletsbin.dir</code> annotated for reference.
Line 21: Line 21:
 
| 0x00 || 4 || Size of DIR file
 
| 0x00 || 4 || Size of DIR file
 
|-
 
|-
| 0x04 || 4 || Number of structs
+
| 0x04 || 4 || Number of entries
 
|-
 
|-
 
|}
 
|}
  
 
{| class="wikitable"  
 
{| class="wikitable"  
|+ Mock-directory struct
+
|+ Mock-directory entry
 
! Offset || Size || Purpose
 
! Offset || Size || Purpose
 
|-
 
|-
Line 43: Line 43:
  
 
=== ARC ===
 
=== ARC ===
Here is <code>/dataDir/archives/pelletsbin.arc</code> annotated to demonstrate searching the BLOB archive. Without the DIR file as a guide, the only way you could possibly differentiate between files would be with the <code>0xCC</code> padding between files.
+
Here is <code>/dataDir/archives/pelletsbin.arc</code> annotated to show how mapping the ARC file would look like with information from the corresponding DIR. Without the DIR file as a guide, the only way you could possibly differentiate between files would be with the <code>0xCC</code> padding between files.
  
 
[[File:ARC_file_documentation.png]][[File:Minty_Meeo_Documentation_Legend.png]]
 
[[File:ARC_file_documentation.png]][[File:Minty_Meeo_Documentation_Legend.png]]
  
 
==== <code>0xCC</code> padding ====
 
==== <code>0xCC</code> padding ====
Files in the ARC are always padded out to have an amount of bytes that is a multiple of 32. These bytes are not included in the file size recorded in the DIR for each file. It's like this because it's an optimization for loading data into the Gamecube's data registers.
+
Files in the ARC always have padding to nearest multiple of 32, or 0x20. This is because data loading into the Gamecube's register's in multiples of 32 is an optimization.
  
 
== Extra information ==
 
== Extra information ==
Every paired ARC+DIR file archive has an uncompressed copy elsewhere on the disc.
+
Every paired Bundled Archive file archive has an uncompressed copy elsewhere on the disc.
* In <code>/dataDir/tekis</code> and <code>/dataDir/bosses</code>, a file named <code>arc_all.ini</code> can be found.  These INI files holds nothing, but give us a hint at how the developer's archiving tool functioned.
+
* In <code>/dataDir/tekis</code> and <code>/dataDir/bosses</code>, a file named <code>arc_all.ini</code> can be found.  These INI files holds nothing, but are an indicator to the archiving tool to bundle the directory's contents.
 
* It is unknown if every file in the archives is identical to its uncompressed sister file.
 
* It is unknown if every file in the archives is identical to its uncompressed sister file.
 
In the [[OpenGL / Dolphin System]], many files are not read from the paired ARC+DIR file archives like they are in the final game, and are instead read from the un-archived versions.   
 
In the [[OpenGL / Dolphin System]], many files are not read from the paired ARC+DIR file archives like they are in the final game, and are instead read from the un-archived versions.   
  
Paired ARC+DIR file archives that have "bin" in their filename curiously only contain the '''BIN''' files from their respective directory.
+
Bundled Archives that have "bin" in their filename only contain the '''BIN''' files from their respective directory.
  
 
The <code>log.txt</code> files left in the root of [[Pikmin|''Pikmin'' versions USA Demo, USA v1.00, and JPN v1.02]] detail the plugin "[[OpenGL / Dolphin System|plugTexConv]]" creating a paired ARC+DIR file archive.
 
The <code>log.txt</code> files left in the root of [[Pikmin|''Pikmin'' versions USA Demo, USA v1.00, and JPN v1.02]] detail the plugin "[[OpenGL / Dolphin System|plugTexConv]]" creating a paired ARC+DIR file archive.
  
{{Credits|[[User:Minty_Meeo|Minty_Meeo]]}}
+
{{Credits|[[User:Minty_Meeo|Minty_Meeo]], [[User:Patrick_Anton|Ambrosia]]}}
  
 
[[Category:Pikmin]]
 
[[Category:Pikmin]]
 
[[Category:File formats]]
 
[[Category:File formats]]

Revision as of 19:47, 28 December 2019

ARC files (short for Archive) and DIR files (internally named Bundle as to not confused with BUN files) make up one of the games proprietary data storage formats. The DIR file contains all necessary information about the data in the ARC file, which stores the aforementioned data.

With minimal metadata, the Bundled Archive file format was used to reduce load time by packing several files into one. The purpose it serves is different from the bundle archive, however, as standalone files within the archive can be read independently of the files before it. This makes the Bundled Archive file format useful for archiving large directories with varied purposes, as to it's name.

File structure

As mentioned before, the paired ARC+DIR file archive consists of two files: the mock-directory (DIR) and the BLOB archive (ARC). Think of the mock-directory as a map to navigate through the BLOB archive.

DIR

DIR files consist of a header followed by a repeating entry for each mock-filepath.

Here is /dataDir/archives/pelletsbin.dir annotated for reference.

DIR file documentation.pngMinty Meeo Documentation Legend.png

Header
Offset Size Purpose
0x00 4 Size of DIR file
0x04 4 Number of entries
Mock-directory entry
Offset Size Purpose
0x00 4 Offset of file in ARC
0x04 4 Size of file in ARC
0x08 4 Mock-directory string length
0x0C 0x08 value Mock-directory path

Mock-directory

The DIR file emulates filepaths. When a file is unpacked in-game, it is assumed that whatever code is responsible treats the entry in the DIR file as a legitimate filepath on the disc, starting from the root.

ARC

Here is /dataDir/archives/pelletsbin.arc annotated to show how mapping the ARC file would look like with information from the corresponding DIR. Without the DIR file as a guide, the only way you could possibly differentiate between files would be with the 0xCC padding between files.

ARC file documentation.pngMinty Meeo Documentation Legend.png

0xCC padding

Files in the ARC always have padding to nearest multiple of 32, or 0x20. This is because data loading into the Gamecube's register's in multiples of 32 is an optimization.

Extra information

Every paired Bundled Archive file archive has an uncompressed copy elsewhere on the disc.

  • In /dataDir/tekis and /dataDir/bosses, a file named arc_all.ini can be found. These INI files holds nothing, but are an indicator to the archiving tool to bundle the directory's contents.
  • It is unknown if every file in the archives is identical to its uncompressed sister file.

In the OpenGL / Dolphin System, many files are not read from the paired ARC+DIR file archives like they are in the final game, and are instead read from the un-archived versions.

Bundled Archives that have "bin" in their filename only contain the BIN files from their respective directory.

The log.txt files left in the root of Pikmin versions USA Demo, USA v1.00, and JPN v1.02 detail the plugin "plugTexConv" creating a paired ARC+DIR file archive.

Credits: Minty_Meeo, Ambrosia