The model file format of Ultimate 3D can be used to save exact copies of model objects to files with the extension *.u3d. For efficient loading the data is saved in a form, which is very close to the final data Ultimate 3D needs. The file format uses a simple chunk based structure and is binary. This paper describes its structure and the single chunks of unencrypted files with version 1.0 in detail, to give you the possibility to import or export it through your applications. There is a newer version of this file format referred to as v. 2.0, which is used by Ultimate 3D 3.0. It is not supported by Ultimate 3D 2.x, but the Blender plugin can export to it. Since it is cleaner, less specific to Ultimate 3D and more extendible you should prefer it, if your work is not related to Ultimate 3D 2.x. Its documentation is available on Ultimate3D.org and is a bit more detailed than this one.
To make this description as clear as possible I want to define some terms here, especially the use of data structures and the used coordinate system:
- bool: A 8-bit value, giving a boolean information.
- char: A 8-bit signed integer value in the range from -128 to 127. Usually used to describe ASCII characters.
- BYTE: A 8-bit unsigned integer value in the range from 0 to 255.
- WORD: A 16-bit unsigned integer value in the range from 0 to 65535.
- DWORD: A 32-bit unsigned integer value in the range from 0 to 4294967295.
- float: A 32-bit signed floating point value as defined in IEEE 754-1985.
- String: A null-terminated string. It's one char after the other and the data ends after the first char that equals zero.
- Vector: Three float values giving a position or direction vector. The first one is the x-component, the second one is the y-component, the third one is the z-component. Ultimate 3D uses a left-handed coordinate system. The x-axis describes the axis that points to the right, the y-axis points upwards and the z-axis points into the depth.
- Quaternion: Four float values describing a rotation by giving a quaternion as described by Ken Shoemake.
- Color: Four float values in the range from 0 to 1 giving a color in the order r, g, b, a.
- Triangle: Three WORD values giving three vertex indices, which describe a triangle.
- Matrix: 16 float values describing a 4x4 transformation matrix. The first four values contain the entries of the first line and so on. Vectors get interpreted as column vectors and when they get transformed using a matrix the matrix gets multiplied from the left side.
- CompressedNormal: This is a very special data structure, which is used to save normal vectors efficiently. Since normal vectors are always normalized they can be described by a latitude and a longitude. Sufficient precision for these values can be reached with 8-bit integers (char). For this reason this structure is made up of two char values, the first one describing the latitude, the second describing the longitude. For the latitude an unfortunate mistake (see the according forum topic) lead to the fact that the available values are not used efficiently. Since this was not noticed before Ultimate 3D 2.x was discontinued it has never been fixed. It is corrected in version 2.0 of this file format. For this reason -31.75 refers to -Pi/2 and 31.75 refers to Pi/2 (use the conversion constants below). For the longitude -127 refers to -Pi and 127 refers to Pi. Through this method the memory needed to describe a normal gets reduced from 12 bytes to 2 bytes. Even though the mistake reduces the precision the loss of information is still not noticeable. Here is some exemplary C++ code on how to convert between Vector and CompressedNormal.
CompressedNormal(CVector Vector){
Vector.Normalize();
float LatitudeF=-asinf(Vector.y);
float LongitudeF=atan2(Vector.x,Vector.z);
Latitude=static_cast<char>(LatitudeF*20.212678f);
Longitude=static_cast<char>(LongitudeF*40.4253555f);
}
Vector Result;
float LatitudeF=static_cast<float>(Latitude)*0.0494739f
float LongitudeF=static_cast<float>(Longitude)*0.02473695f;
float CosLatitude=cos(LatitudeF);
Result.x=CosLatitude*sin(Longitude);
Result.y=-sin(Latitude);
Result.z=CosLatitude*cos(Longitude);
return Result;
} |
|---|
Last but not least here are some notations that will be used below. A construct in the form of <DataType1;DataType2;...;DataTypeN> implies a new data type, that is a structure made up of one occurrence of DataType1 at it's beginning and one occurrence of DataType2 after that and so on. Square brackets in the form [x] behind a variable imply that this is an array made up of x values of the given type. If a variable identifier begins with "n", this implies that the variable gives the count of something. If a variable identifier begins with a "p", this implies that it's an array. If a variable identifier begins with an "i", this implies that it is an index.
The chunk structure of the file format is very simple. Every chunk is just a collection of binary data, with varying size, that describes a particular logical object. All chunks begin with a short identification string, which is followed by the individual data of the chunk. After the end of every chunk (despite the last one) a new chunk begins immediately, without any data inbetween. The first chunk is the header. It contains basic information about the model and gives information about the chunks that will follow. In particular it says how often the other chunks will appear. After that all meshes are described, then the materials follow and finally the bones get described.
The first bytes in an unencrypted *.u3d file always contain the header. Here's a description of the data that's given by it in the order it occurs in.
char U3DModelFile[13]
This string identifies this file as Ultimate 3D model file. If it's "U3DModelFile" this file is a Ultimate 3D model file of version 1.0, otherwise it is not. The string is null-terminated.
float FileVersion
This floating point value gives the file version. Currently the only existing file version is 1.0.
WORD nMesh
The total number of meshes that are saved within this Ultimate 3D model file. This is the number of meshes per frame multiplied by the number of LoDs and if no skeletal animation is used multiplied by the number of frames (nMeshesPerFrame*nLOD*(nFrame*(nBone==0)+(nBone!=0))). It equals the number of mesh chunks in this file. If skinning is used this value has to equal one! This means that there has to be exactly one mesh if vertex skinning is used.
WORD nMeshesPerFrame
The number of different meshes that are needed to render the model in every frame. For example for a model with a skeleton that has ten meshes appended to it, but only five different meshes, this value would be five.
DWORD nFrame
The number of frames the animation of this model is made up of. Make sure that you don't set up zero for this, if you are exporting. Every model has at least one frame.
WORD nLOD
The number of different levels of detail (LoD) for this model. Just like nFrame this value has to be at least one.
WORD nMaterial
The number of different materials for this model, which equals the number of material chunks. This must not be zero, too.
WORD nBone
The number of bones for this model, which equals the number of bone chunks.
float CelShadingStrength
The width of the outline that is to be created through cel-shading.
Color CelShadingColor
The color of the cel-shading outline.
bool AlphaBlending
Whether alpha blending is to be used or not. You should set this always to true, if you are exporting.
bool VertexTweening
Whether vertex tweening is to be used to smooth the animation of this model. Setting this to true is only valid if the model doesn't use skeletal animation (meaning nBone==0) and has the same number of vertices and triangles for every frame of the frame based animation.
float pLODLevelOfDetail[nLOD-1]
The level of detail for every LoD that's not the highest LoD. The array begins with information about the second highest LoD. This is a floating point value in the range from 0 to 1, where 0 means zero triangles and 1 means the triangle count of the highest LoD.
float pLODCameraDistanceSq[nLOD-1]
The square of the distance at which Ultimate 3D should start using a particular LoD for every LoD which is not the highest LoD. The array begins with information about the second highest LoD.
That's all for the header. The next bytes describe the first mesh.
The mesh chunks are placed one after the other immediately after the header. They occur Header.nMesh times and contain all data that is related to the geometry of the model. For this reason the mesh chunk is the most complex chunk. Here's a list of all the data it can include:
char MeshID[10]
This string can be used to identify this mesh chunk. It should always be a null-terminated "$U3D_MESH".
String Name
The name of this mesh which is used for some error messages.
WORD Frame
If this model uses frame based animation this value implies the frame this mesh is a part of. If the model uses skeletal animation or doesn't have an animation it should be zero.
float LOD
The level of detail of this mesh. This value multiplied by the triangle count of the mesh with the highest LoD results in the triangle count of this mesh.
float NormalScalar
A factor to multiply the normals with. If one of the bones uses a negative scaling to mirror this mesh this value should be -1. Usually it's 1.
bool ValidInverseTangentSpaceMatrices
Whether this mesh has two additional three dimensional texture coordinate sets and modified normals, which describe inverse tangent space matrices for every vertex. If you are exporting *.u3d files you should always set this to false.
WORD nVertex
The number of vertices the geometry of this mesh is made up of.
WORD pTexCoordDimensions[8]
Every mesh can have up to eight different texture coordinate sets. Every texture coordinate set can have zero to four dimensions (meaning that it's made up of zero to four float values). This array gives the dimension of every texture coordinate set. The common case is that the first value is two and all other values are zero.
WORD nSkinWeight
The maximum number of skin weights per vertex minus one. This value should be either zero, one or three. If it would be two Ultimate 3D wouldn't have the required per pixel lighting and shadowing shaders ready when they are needed.
BYTE iMaxBlendingMatrix
The maximum of the indices of the bones that influence this mesh. For example if this mesh is influenced by bone 1, 0, 14 and 5 this value had to be 14.
WORD nShadowVolumeVertex
The number of vertices for the shadow optimized geometry. If you are exporting a *.u3d file you should simply pass zero for this and stop thinking about it.
WORD nShadowVolumeTriangle
The number of triangles for the shadow optimized geometry. If you are exporting a *.u3d file you should simply pass zero for this and stop thinking about it.
float pShadowVolumeVBuffer[(6+nSkinWeight+(nSkinWeight!=0))*nShadowVolumeVertex]
The vertices of the shadow optimized geometry. It begins with the position of the first vertex, continues with it's normal and after that the skin weights (nSkinWeight*float) and - if nSkinWeight is bigger than zero - the skin weight indices (4*BYTE) follow. Then it continues with the second vertex. If you import a *.u3d file you should better skip this data, the purpose it's meant to be used for is very special.
Triangle pShadowVolumeIBuffer[nShadowVolumeTriangle]
The index buffer of the shadow optimized geometry.
Vector pPosition[nVertex]
The position of every single vertex in mesh space.
CompressedNormal pNormal[nVertex]
The normal of every single vertex in mesh space.
bool UseTextureCoordinateBuffer
Whether the texture coordinates are meant to be handled separately or not. You should set this to false if you are exporting and ignore it if you are importing. Ultimate 3D uses this option to save memory when it's working with *.md2 files.
bool TextureCoordinateOwned
If UseTextureCoordinateBuffer is true it may be that this value is false. In this case the data sets below, which give the texture coordinates, don't exist. This can only happen if frame animation is used. In this case the texture coordinates from the first frame need to be used for this mesh. The common case is that this value is true.
float ppTextureCoordinateSet0[nVertex][pTexCoordDimensions[0]]
All texture coordinates of the first texture coordinate set for all vertices.
float ppTextureCoordinateSeti[nVertex][pTexCoordDimensions[i]]
It works the same way for the other seven possible texture coordinate sets.
float ppSkinWeight[nVertex][nSkinWeight]
The skin weights for every vertex. The last skin weight, which is not given by any of these floats, gets calculated by taking one minus the sum of the given skin weights. This way the sum of all weights is always one.
BYTE pSkinWeightIndex[nVertex][4*(nSkinWeight!=0)]
The indices of the bones a vertex is effected by for every single vertex. If skinning is used it's always four indices per vertex. Some of them may not be used dependent on the value of nSkinWeight.
WORD nTriangle
The number of triangles this mesh is made up of.
bool TrianglesOwned
This value is very similar to TextureCoordinateOwned. If it's false frame animation is used and this mesh needs to use the triangle indices of the first mesh in the animation. In this case the next two data sets don't exist. Usually it's true.
Triangle pTriangle[nTriangle]
The vertex indices for all triangles.
WORD piTriangleMaterial[nTriangle]
The index of the used material for every triangle.
That's all about the meshes. After the Header.nMesh mesh chunks the Header.nMaterial material chunks follow immediately.
The material chunks are placed immediately after the mesh chunks and they occur Header.nMaterial times. One could call the material chunk the most complex chunk of the Ultimate 3D model file format, since it can contain shader effects with all information that is needed for them and texture files of any type. Documenting these data structures would be very complicated. For this reason this documentation describes only materials that don't have their textures exported and don't contain shader effects:
char MaterialID[9]
This string identifies this chunk as a material chunk. It should be always "$U3D_MAT" with a null terminating it.
Name
The name of this material. It functions as identifier for functions like GetMaterialIndex(MaterialName).
Color Ambient
The ambient lighting color of this material. The brighter it is the stronger is the effect of ambient light on this model.
Color Diffuse
The diffuse color of this material.
Color Specular
The specular color of this material. Currently Ultimate 3D ignores this, but that's going to change soon.
Color Emissive
The emissive color of this material.
float SpecularPower
The exponent that's used to intensify the specular highlights.
WORD nTexture
The number of textures (resp. texture stages) that is used by this material.
DWORD pColorOp[8]
The color operation that is used for this texture stage. It is a value from the D3DTEXTUREOP enumerated type of Direct3D.
DWORD piTexCoordSet[8]
The index of the texture coordinate set that's used for each stage. A common choice would be i for stage i or 0 for all stages.
bool ExportResources
Whether texture and shader files are contained within this file or not. If it's true you should not load any textures and scan forward to the next "$U3D_MAT".
<bool[DWORD,String,WORD,WORD,String]> pTextureData[8]
The texture data for all textures (assuming that ExportResources is false). The bool at the beginning says whether this stage uses a texture. If it's false the other information, which is in square brackets above, is not available and the next byte gives information about the next texture. If it's true the DWORD gives the type of the texture, which should be zero (otherwise you should better skip to the next material). The String after that is a unique identifier for this texture. Usually this is the file name with path. The two WORD values give the width and the height of the texture then and the String gives the file name with relative or absolute path. If the file name doesn't have a path Ultimate 3D will search the texture in it's currently setup texture folder when loading.
bool EffectUsed
Whether this material uses a shader effect. If this is true skip to the next material.
This part isn't that pretty since there are many cases in which data needs to be skipped when importing *.u3d files. Anyway the chunk identifier "$U3D_MAT" makes skipping quite easy, so this shouldn't be a that big problem. There are worth things than being unable to load textures sometimes. The set of material chunks is followed by the last set of chunks, the bone chunks.
The bone chunks are placed immediately after the material chunks and occur Header.nBone times. They describe the complete skeleton including the animation and which meshes are appended to it. Here's a list of the data a bone chunk can hold:
char BoneID[10]
This string identifies this chunk as a bone chunk. It should always be a null-terminated "$U3D_BONE".
String Name
The name of this bone. It functions as identifier for functions like GetBoneIndex(BoneName).
WORD iParent
The index of the bone that's the parent of this bone. If this bone is a root bone (there can be multiple) this value has to be 65535 (the maximum value, that can be saved by a 16-bit integer).
float BoneFrame
If this value is positive the frame value given by it will be used for this bone instead of the frame value of the parent bone. If it's negative the frame value of the parent will be used. Since that's the usual behavior you should set up -1 for this.
bool PassOnBoneFrame
Whether the child frame should use the same frame value as this frame.
WORD nMesh
The number of meshes that are appended to this bone. If skinning is used this has to be one for all bones.
WORD piMesh[nMesh]
The indices of the meshes that are appended to this bone.
Matrix pMeshTransformation[nMesh]
The mesh to bone space transformation matrix for every mesh that is appended to this bone. To get the final transformation of a mesh this matrix gets multiplied by the bone transformation from the right side.
WORD nScalingKey
The number of scaling keys. If you want just a default key with no scaling this can be zero.
WORD Unused
Two bytes of unused data. The content has no effect.
<WORD;WORD;Vector> pScalingKey[nScalingKey]
The scaling keys for this bone. The first WORD gives the frame the key is at, the second WORD is absolutely unused, the Vector gives the scaling among each of the three axes in bone space.
WORD nTranslationKey
The number of translation keys. If you want just a default key with no translation this can be zero.
WORD Unused
Two bytes of unused data. The content has no effect.
<WORD;WORD;Vector> pTranslationKey[nTranslationKey]
The translation keys for this bone. The first WORD gives the frame the key is at, the second WORD is absolutely unused, the Vector gives the translation in parent bone space.
WORD nRotationKey
The number of rotation keys. If you want just a default key with no rotation this can be zero.
WORD Unused
Two bytes of unused data. The content has no effect.
<WORD;WORD;Quaternion> pRotationKey[nRotationKey]
The rotation keys for this bone. The first WORD gives the frame the key is at, the second WORD is absolutely unused, the Quaternion describes the rotation in parent bone space.
This is all information that is needed to load the bones. Since loading animations is always a quite complicated task I want to give some additional details on the relations of the given transformation data at this point. Sorry for the pseudo code formulas, but I guess it's quite understandable this way. From the transformation keys you can compute a rotation, a translation and a scaling. The bone to parent bone space matrix is defined as BoneToParentBoneSpace=Scaling*Rotation*Translation. Assuming that the parent bone already has a parent bone to world space matrix the bone to world space matrix is defined as BoneToWorldSpace=BoneToParentBoneSpace*ParentBoneToWorldSpace (it's defined recursive). With the mesh to bone space matrices given in the bone chunk you can easily get the mesh to world space matrices through pMeshToWorldSpace[i]=pMeshTransformation[i]*BoneToWorldSpace then.
After the last bone chunk the *.u3d file is over.
I hope this documentation is clear enough. I have gone through the frustration of being forced to guess around how a particular part of a model file format is to be interpreted due to an ambiguous documentation often enough. For this reason anybody who has problems to understand a part of this documentation or thinks that there's a mistake in it should feel free to contact me at any time. You can contact me by email, on the forum of Ultimate 3D or via the contact form on Ultimate3D.org.
© Christoph Peters. Some rights reserved.