Tibia 11 - .dat File Structure

[Jo3Bingham]

https://tibiapf.com/showthread.php?101-dat-Structure

Here you can see my progress in understanding the new .dat file structure. My 4th post (Update #3) contains a basically-complete breakdown of the structure. I still have a few flags to find, and some unknown values to figure out, but, for the most part, I have the whole structure decoded.

[blackd]

Thank you for sharing. I was about to write that I already finished parsing all the properties there, obtaining same property info than what I obtained with old tibia.dat 10.98

For me, the hardest part was solving the problem of this smart numeric language:
64 = 100
65 = 101
...
7F = 127
80 01 = 128
81 01 = 129
...
FE 01 = 254
FF 01 = 255
80 02 = 256
81 02 = 257
82 02 = 258
...
FF 02 = 383
80 03 = 384
...
80 7F = 16256
FF 7F = 16383
80 80 01 = 16384

This is my current function to read them properly:



Public Function readProtobufNumber(ByRef arr() As Byte, ByVal pos As Long, ByRef positionsRead As Long) As Long
Dim num As Long
Dim b1 As Byte
Dim b2 As Byte
Dim b3 As Byte
Dim b4 As Byte
Dim b5 As Byte
On Error GoTo gotErr
b1 = arr(pos)
num = CLng(b1)
If (b1 < &H80) Then
positionsRead = 1
num = CLng(b1)
Else
b2 = arr(pos + 1)
num = num + (128 * (CLng(b2) - 1))
If (b2 < &H80) Then
positionsRead = 2
Else
b3 = arr(pos + 2)
num = num + (16384 * (CLng(b3) - 1)) ' 16384 = 128*128
If (b3 < &H80) Then
positionsRead = 3
Else
b4 = arr(pos + 3)
num = num + (2097152 * (CLng(b4) - 1)) ' 2097152 = 16384*128
If (b4 < &H80) Then
positionsRead = 4
Else
b5 = arr(pos + 4)
num = num + (268435456 * (CLng(b5) - 1)) ' 268435456 = 2097152*128
positionsRead = 5
End If
End If
End If
End If
readProtobufNumber = num
Exit Function
gotErr:
readProtobufNumber = -2
End Function

' Example: testProtobufNumber ("80 80 01")
Public Sub testProtobufNumber(ByVal str As String)
Dim arr() As String
Dim arrb() As Byte
Dim res As Long
Dim i As Integer
Dim positions As Long
Dim tam As Long
str = Trim$(str)
arr = Split(str, " ")
tam = UBound(arr)
ReDim arrb(tam)
For i = 0 To UBound(arr)
arrb(i) = CByte(CLng("&H" & arr(i)))
Next i
res = readProtobufNumber(arrb, 0, positions)
Debug.Print str & " = " & CStr(res) & " (used " & CStr(positions) & " positions)"
End Sub


Problem solved.
After understanding that, the rest should be a piece of cake.
The flags there are almost the same than Tibia 10.98. Very few differences. It is just funny game of matching tibia.dat flags with flags at new .dat

My full parser: http://blackdtools.net/tibia11parser.txt

[Jo3Bingham]

That's an odd numerical system... Thanks for the info, though. I appreciate it.

[blackd]

After some further investigation I discovered this is part of a system called protobuf.

I also made a new VB6 function to convert a 4 bytes number into a protobuf number (the way back). It was only a subproduct of one of my recent investigations on Tibia and maybe it is not really usefull for me, however -maybe- someone will find it usefull:



Public Sub getProtobufBytesFrom4BYTESNUMBER(ByRef arr() As Byte, ByVal number As Long)
Const cte1 As Long = 128
Const cte2 As Long = 16384 ' 128 * 128
Const cte3 As Long = 2097152 ' 128 * 128 * 128
Const cte4 As Long = 268435456 ' 128 * 128 * 128 * 128
If number < cte1 Then
ReDim arr(0)
arr(0) = CByte(number)
ElseIf number < cte2 Then
ReDim arr(1)
arr(1) = CByte(number \ cte1)
number = number - (arr(1) * cte1)
arr(0) = CByte(cte1 + number)
ElseIf number < cte3 Then
ReDim arr(2)
arr(2) = CByte(number \ cte2)
number = number - (arr(2) * cte2)
arr(1) = CByte(cte1 + (number \ cte1))
number = number - ((arr(1) - cte1) * cte1)
arr(0) = CByte(cte1 + number)
ElseIf number < cte4 Then
ReDim arr(3)
arr(3) = CByte(number \ cte3)
number = number - (arr(3) * cte3)
arr(2) = CByte(cte1 + (number \ cte2))
number = number - ((arr(2) - cte1) * cte2)
arr(1) = CByte(cte1 + (number \ cte1))
number = number - ((arr(1) - cte1) * cte1)
arr(0) = CByte(cte1 + number)
Else
ReDim arr(4)
arr(4) = CByte(number \ cte4)
number = number - (arr(4) * cte4)
arr(3) = CByte(cte1 + (number \ cte3))
number = number - ((arr(3) - cte1) * cte3)
arr(2) = CByte(cte1 + (number \ cte2))
number = number - ((arr(2) - cte1) * cte2)
arr(1) = CByte(cte1 + (number \ cte1))
number = number - ((arr(1) - cte1) * cte1)
arr(0) = CByte(cte1 + number)
End If
Debug.Print arrayToString(arr)
End Sub

Public Sub testGetProto(ByVal number As Long)
Dim arr() As Byte
getProtobufBytesFrom4BYTESNUMBER arr, number
End Sub