Difference between revisions of "Paired ARC+DIR file"

From Pikmin Technical Knowledge Base
Jump to navigation Jump to search
m (Minty Meeo moved page ARC file (Pikmin) to Pikmin ARC+DIR File Pairs: A massive rewrite is necessary, and this old page name does not accurately describe what needs to be described.)
(A complete rewrite to better tell the tale of the Paired ARC+DIR Files archive, as well as finally give them a name.)
Line 1: Line 1:
−
Pikmin 1 has a very stupid ARC format that isn't compatible with LunaBoy's RARC tools and makes everyone very sad.
+
'''ARC''' files (likely short for '''Archive''') and '''DIR''' files (technically short for '''Bundle''', though '''Directory''' makes more sense and avoids conflict with [[Pikmin BUN Files|'''BUN''' Files]]) are two peas in a pod, together representing one of the many Binary Large Object (BLOB) archive formats used in Pikmin 1.  Together, they will henceforth be referred to as the ''Paired ARC+DIR Files'' archive.
 +
 
 +
With minimal metadata, {{unsure|the Paired ARC+DIR Files archive was likely used to reduce load time by packing several files into one}}.  The purpose it serves is different from the [[Pikmin BUN Files|Bundle]] archive, however, {{unsure|as standalone files within the archive can be unpacked independent of the files before it}}.  This makes the Paired ARC+DIR Files useful for archiving large directories with varied purposes.
  
−
However, that doesn't mean that it is difficult to understand-- quite the contrary.
 
  
 
__TOC__
 
__TOC__
  
−
==The Basics==
 
−
A lot of this is currently theory and unprofessional sounding, but hear me out.
 
−
The developers of Pikmin, unsure of how big of a game they were going to make, made a very simple compression script for their files.  This script would take a directory or subdirectory in <code>root\datadir</code> and create a compressed copy of all the files in <code>root\datadir\archives</code>.  This is the ARC file, it is named after the lowest directory accessed, and simply put, it is every file in that directory smushed together with loads of <code>0xCCCC</code> between them to differentiate.
 
  
−
How many sets of <code>0xCCCC</code>? I don't know yet.
+
==File Structure==
 +
As mentioned before, the Paired ARC+DIR Files archive consists of two files: the mock-directory (DIR) and the BLOB archive (ARC).  Think of the mock-directory as a map to navigate through the BLOB archive.
 +
 
 +
===DIR===
 +
DIR files consist of a header followed by a repeating struct for each mock-filepath.
 +
 
 +
Here is <code>root\dataDir\archives\pelletsbin.dir</code> annotated for reference.
 +
 
 +
[[File:DIR_file_documentation.png]][[File:Minty_Meeo_Documentation_Legend.png]]
 +
 
 +
{| class="wikitable"
 +
|+ Header
 +
! Offset || Size || Purpose
 +
|-
 +
| 0x00 || 4 || Size of DIR file
 +
|-
 +
| 0x04 || 4 || Number of structs
 +
|-
 +
|}
 +
 
 +
{| class="wikitable"
 +
|+ Mock-Directory Struct
 +
! Offset || Size || Purpose
 +
|-
 +
| 0x00 || 4 || Offset of file in ARC
 +
|-
 +
| 0x04 || 4 || Size of file in ARC
 +
|-
 +
| 0x08 || 4 || Mock-Directory String Length
 +
|-
 +
| 0x0C || 0x08 value || Mock-Directory Path
 +
|-
 +
|}
 +
 
 +
====Mock-Directory====
 +
The DIR file emulates filepaths.  When a file is unpacked in-game, it is assumed that whatever code is responsible treats the entry in the DIR file as a legitimate filepath on the disc, starting from the root.
 +
 
 +
===ARC===
 +
Here is <code>root\dataDir\archives\pelletsbin.arc</code> annotated to demonstrate searching the BLOB archive. Without the DIR file as a guide, the only way you could possibly differentiate between files would be with the <code>0xCC</code> padding between files.
 +
 
 +
[[File:ARC_file_documentation.png]][[File:Minty_Meeo_Documentation_Legend.png]]
  
−
Anyway, this isn't all their script would do.  It would also create a partner '''DIR''' file describing the directory path, treating datadir as the root, to each file that was stored in the ARC.  It's sort of like emulating the directory path I think, plus it tells us the order of the files in the ARC file.
+
====<code>0xCC</code> Padding====
 +
For whatever reason, files in the ARC are always padded out to have an amount of bytes that is a multiple of 32.  These bytes are not included in the file size recorded in the DIR for each file, so the exact purpose of this is unknown.  {{unsure|It could be an absolute necessity, or it could be completely unnecessary.}}
  
−
[[File:DIR_file_documentation.png]]
 
  
 +
==Extra Information==
 +
Every Paired ARC+DIR Files archive has an uncompressed copy elsewhere on the disc.
 +
* In <code>root\dataDir\tekis</code> and <code>root\dataDir\bosses</code>, a file named <code>arc_all.ini</code> can be found.  These INI files holds nothing, but give us a hint at how the developer's archiving tool functioned.
 +
* It is unknown if the every file in the archives is identical to its uncompressed sister file.
 +
In the [[OpenGL / Dolphin System]], many files are not read from the Paired ARC+DIR Files archives like they are in the final game, and are instead read from the un-archived versions. 
  
−
==The bloody hell are you getting on about?==
+
Paired ARC+DIR Files archives that have "bin" in their filename curiously only contain the '''BIN''' files from their respective directory.
−
So I have been trying to get Beady Long Legs to leave its arena on console.  I specify "on console" because I have been able to get him to leave his arena in the Windows x86 debugger for 6 1/2 months, but no matter what I did, the edits I did to the files on PC didn't change anything on console.  Well, today I was finally able to get BLL OoA on console/Dolphin by editing the archived version of bosses\kumo\parms.bin that is stored within archives\bosses.arc.  Using bosses.dir as a guide, I was able to locate the archived copy of the parameters file, edit it, and run it on console/Dolphin.  [https://www.youtube.com/watch?v=iXJFQKinoBM You can watch it here.]  The next step is to take what I've described and make a tool, right?  I don't know any good programming languages, and if I'm the one making the tool, get ready to wait another 6 1/2 months.
 
  
 +
The <code>log.txt</code> files left in the root of Pikmin 1 [[Pikmin|versions USA Demo, USA v1.00, and JPN v1.02]] detail the plugin "[[OpenGL / Dolphin System|plugTexConv]]" creating a Paired ARC+DIR Files archive.
  
−
==Summary==
 
−
Why haven't we figured out this format yet we have carbon copies of every file in every ARC present on the disc.  Someone make an unpacker+packer so we can hack this gaem pls and thank.
 
  
 
[[Category:Pikmin]]
 
[[Category:Pikmin]]
 
[[Category:File formats]]
 
[[Category:File formats]]

Revision as of 21:48, 23 April 2018

ARC files (likely short for Archive) and DIR files (technically short for Bundle, though Directory makes more sense and avoids conflict with BUN Files) are two peas in a pod, together representing one of the many Binary Large Object (BLOB) archive formats used in Pikmin 1. Together, they will henceforth be referred to as the Paired ARC+DIR Files archive.

With minimal metadata, the Paired ARC+DIR Files archive was likely used to reduce load time by packing several files into one[unsure]. The purpose it serves is different from the Bundle archive, however, as standalone files within the archive can be unpacked independent of the files before it[unsure]. This makes the Paired ARC+DIR Files useful for archiving large directories with varied purposes.



File Structure

As mentioned before, the Paired ARC+DIR Files archive consists of two files: the mock-directory (DIR) and the BLOB archive (ARC). Think of the mock-directory as a map to navigate through the BLOB archive.

DIR

DIR files consist of a header followed by a repeating struct for each mock-filepath.

Here is root\dataDir\archives\pelletsbin.dir annotated for reference.

DIR file documentation.pngMinty Meeo Documentation Legend.png

Header
Offset Size Purpose
0x00 4 Size of DIR file
0x04 4 Number of structs
Mock-Directory Struct
Offset Size Purpose
0x00 4 Offset of file in ARC
0x04 4 Size of file in ARC
0x08 4 Mock-Directory String Length
0x0C 0x08 value Mock-Directory Path

Mock-Directory

The DIR file emulates filepaths. When a file is unpacked in-game, it is assumed that whatever code is responsible treats the entry in the DIR file as a legitimate filepath on the disc, starting from the root.

ARC

Here is root\dataDir\archives\pelletsbin.arc annotated to demonstrate searching the BLOB archive. Without the DIR file as a guide, the only way you could possibly differentiate between files would be with the 0xCC padding between files.

ARC file documentation.pngMinty Meeo Documentation Legend.png

0xCC Padding

For whatever reason, files in the ARC are always padded out to have an amount of bytes that is a multiple of 32. These bytes are not included in the file size recorded in the DIR for each file, so the exact purpose of this is unknown. It could be an absolute necessity, or it could be completely unnecessary.[unsure]


Extra Information

Every Paired ARC+DIR Files archive has an uncompressed copy elsewhere on the disc.

  • In root\dataDir\tekis and root\dataDir\bosses, a file named arc_all.ini can be found. These INI files holds nothing, but give us a hint at how the developer's archiving tool functioned.
  • It is unknown if the every file in the archives is identical to its uncompressed sister file.

In the OpenGL / Dolphin System, many files are not read from the Paired ARC+DIR Files archives like they are in the final game, and are instead read from the un-archived versions.

Paired ARC+DIR Files archives that have "bin" in their filename curiously only contain the BIN files from their respective directory.

The log.txt files left in the root of Pikmin 1 versions USA Demo, USA v1.00, and JPN v1.02 detail the plugin "plugTexConv" creating a Paired ARC+DIR Files archive.

Retrieved from "https://pikmintkb.com/w/index.php?title=Paired_ARC%2BDIR_file&oldid=1498"