Nikon D5100 Video Beta Testers Wanted

The Nikon Hackers team would like to announce the release of the first hack for the D5100 (also D3100 & D7000): Removing time limit restrictions on video recording.

The web-browser cross platform patch tool

To use:

  1. Download the D5100/D3100/D7000 firmware
  2. Use Web Tool above
  3. Copy the resulting output to your SD card and remove the “.patched” extension
  4. Upgrade firmware from D5100 menu, as per the normal Nikon instructions

This should not brick your camera, but if it does I and the other Nikon Hacker team members will not have/take responsibility for any damages or liability. I have this firmware on my camera, and have taken a 25 minute video with it, but result may vary. Also damages that may occur due to camera operation times longer than specified by original manufacturer are your responsibility.

The Nikon Hacker forums is the place for discussing this patch, future patchs, or just to get involved.

We would like to invite any interested developers to help progress the firmware decoding process.

Soon we will be releasing the same patch for D3100 and D7000 camera’s but we have not tested these patches at all, so we are looking for VERY KEEN Alpha testers to try it out these changes. Contact me directly (simeon@simeonpilgrim.com) if you are interested and don’t mind possibly bricking your camera. If you don’t know what bricking means, don’t apply. All three tested and released.

[update] If you apply the patch please report results here or On this Nikon Hackers Forum  post

[Update 31/3] v1.1 has been released with support for D3100 camera’s
[Update 2/4] v1.2 has been released with support for D7000 camera’s

Interested in more, come join the us at Nikon Hacker, or use the Online Patch Tool (Help)

Nikon D5100 Video Record Time Limit Found

I’ve found the memory location of the video recording limit (20 minutes on the D5100) and change it to 1 minute to allow for faster testing.

Here’s a really exciting video where is stops record after 1 minute with no user involvement.

[Update 40 minutes later]

The D3100 10 minute limit has been located, and the D7000 20 minute limit has also been located.

[Update 30th March]

Here’s a screen snap of a 25:59 minute video (1080P 30fps @18mbps) I just recorded on my D5100. At lowest quality settings (424 30fps @4mbps) it says I can record 59minutes of video, so I assume the it doesn’t display hours because 1h 59m @4mbp is 3.5GB which is around the size of my ~26m video.

25min movie
25min movie

[Update 1st April]
Patch available from here: Nikon D5100 Video Beta Testers Wanted

[Update 24th April] A web-browser cross platform patch tool is also available.

Interested in more, come join the us at Nikon Hacker, or use the Online Patch Tool (Help)

Nikon J1/V1 Firmware decoded

I started looking at the J1/V1 firmware last weekend.

As compared to the DSLR firmware the XOR pattern is different, but using knowledge from the D5100 firmware, I made very quick progress getting past the first two XOR layers, the third was a little tricky.

Then yesterday, I got an email from Петр Кудинов that showed where in the D5100 firmware the XOR tables were stored. With this new insight (that they were stored verse calculated) I proceeded to search for what I had already found of the V1 XOR pattern in that firmware, and struck gold.

Here is the firmware based patterns used to decode the V1 and J1 firmware:

static byte[] Xor_Ord1_5 = {
    0xBB, 0x85, 0x12, 0xA7, 0xD7, 0x11, 0x15, 0x30, 0x53, 0x5C, 0x72, 0xCA, 0x61, 0x9F, 0xA3, 0xAC,
    0x2D, 0xC7, 0x05, 0x9C, 0xDF, 0x22, 0x37, 0xF5, 0x93, 0x6F, 0x24, 0x6B, 0x98, 0xB2, 0x0D, 0x44,
    0x8B, 0xCD, 0x3D, 0x33, 0x65, 0xF3, 0xF7, 0x52, 0x89, 0xB7, 0x34, 0x26, 0x79, 0x8F, 0xC1, 0x84,
    0x87, 0x07, 0x3A, 0xE2, 0xE0, 0x0C, 0xA1, 0xF0, 0xA0, 0x77, 0x4A, 0x7E, 0xFF, 0x25, 0xB6, 0x9D,
    0x4F, 0x36, 0xF4, 0xC8, 0xA9, 0xA8, 0x70, 0x00, 0x99, 0xC3, 0x32, 0x2A, 0x1F, 0xDA, 0x9B, 0x64,
    0xEA, 0xBA, 0x63, 0xC2, 0x23, 0xAF, 0xBF, 0x2C, 0x39, 0x16, 0x9E, 0x6D, 0xDC, 0x31, 0x02, 0x19,
    0x2F, 0x47, 0xAE, 0xE3, 0x5B, 0x74, 0x0F, 0x71, 0xD4, 0xFE, 0x50, 0xD6, 0x3E, 0xD0, 0x18, 0xA4,
    0xE8, 0x29, 0x80, 0x5A, 0x7D, 0xE5, 0x45, 0xE1, 0x62, 0x6C, 0xE6, 0xB5, 0x9A, 0x78, 0xA5, 0x7B,
    0x03, 0x81, 0x27, 0xB0, 0x06, 0xF1, 0xE7, 0x8E, 0xD2, 0xD9, 0x1E, 0x3F, 0x5E, 0x95, 0xED, 0xFA,
    0x20, 0x90, 0xEE, 0xFD, 0xDD, 0x4E, 0xDB, 0x6A, 0x82, 0x7A, 0x56, 0x1B, 0x8A, 0x1C, 0x09, 0x57,
    0x6E, 0x2B, 0xE4, 0x04, 0xAB, 0x67, 0xDE, 0xF6, 0x1D, 0x2E, 0x46, 0x86, 0x8C, 0xCB, 0xB8, 0x14,
    0x66, 0x10, 0xD1, 0xF2, 0x8D, 0x08, 0x54, 0xB4, 0x42, 0xF8, 0x49, 0xD5, 0x75, 0xBE, 0x76, 0x68,
    0x51, 0x7C, 0xEF, 0x96, 0xCE, 0x0E, 0xE9, 0xEB, 0x41, 0x28, 0x59, 0xEC, 0xB1, 0x13, 0x91, 0x40,
    0x0A, 0xCF, 0x5D, 0x01, 0xC9, 0x3B, 0xC0, 0xAA, 0x88, 0xC5, 0x92, 0x38, 0x4B, 0x5F, 0xD3, 0xA6,
    0x69, 0xA2, 0x35, 0x58, 0xFB, 0xC6, 0x0B, 0x17, 0x7F, 0x21, 0x3C, 0x83, 0xBC, 0xCC, 0x4D, 0x97,
    0xC4, 0x60, 0x48, 0xF9, 0xBD, 0x73, 0xB9, 0xB3, 0x1A, 0x55, 0xD8, 0x4C, 0xAD, 0xFC, 0x43, 0x94
};

static byte[] Xor_Ord2_5 = {
    0xB6, 0x0A, 0x9C, 0xBF, 0x54, 0x5C, 0x80, 0x8F, 0xFE, 0x1E, 0xF9, 0x3A, 0x8C, 0xEA, 0x78, 0x2A,
    0x01, 0xF7, 0x56, 0x38, 0x39, 0xE5, 0x72, 0xE6, 0x44, 0x0C, 0x41, 0x73, 0x82, 0x34, 0xCD, 0xC2,
    0x14, 0x37, 0xC7, 0xBD, 0xD3, 0xF3, 0x29, 0xAA, 0x8D, 0xC5, 0xE4, 0xEB, 0x9D, 0xE9, 0xFB, 0xF1,
    0x1F, 0x7D, 0x07, 0x93, 0x53, 0x95, 0x4A, 0x24, 0x05, 0x4D, 0xCE, 0x7E, 0x7F, 0x28, 0x22, 0xC8,
    0x6E, 0x40, 0x55, 0xE3, 0x65, 0xC3, 0x3B, 0x0F, 0xE2, 0xDC, 0xED, 0xBC, 0x1B, 0x61, 0xEF, 0x0B,
    0x92, 0x99, 0x1D, 0x64, 0x8E, 0x7C, 0xE1, 0x60, 0xAF, 0xA6, 0x85, 0xEC, 0x4C, 0xB8, 0x69, 0x23,
    0xDB, 0xBA, 0xCC, 0xC6, 0xEE, 0x0D, 0x27, 0x43, 0xF8, 0x11, 0x32, 0xFC, 0x75, 0xD1, 0xA1, 0x20,
    0x71, 0x63, 0x88, 0x2B, 0xD9, 0x2E, 0x89, 0x68, 0x3C, 0xF4, 0xDF, 0x33, 0xFD, 0xAC, 0x6C, 0xB3,
    0x19, 0xC1, 0x10, 0x57, 0x17, 0xFA, 0x79, 0xCF, 0x13, 0xF2, 0x86, 0x35, 0x51, 0xB1, 0xD5, 0xCB,
    0xA3, 0xAE, 0x94, 0x03, 0x5F, 0xD8, 0x6B, 0x31, 0x74, 0x81, 0xBB, 0x3D, 0x15, 0x16, 0xF6, 0xB5,
    0x4B, 0xF5, 0x77, 0x6F, 0x49, 0xA8, 0xA7, 0xCA, 0xAD, 0x02, 0x9A, 0x5E, 0x59, 0x25, 0x98, 0x62,
    0x1A, 0xB7, 0x9F, 0x2F, 0xA0, 0x76, 0x6D, 0xB4, 0x47, 0xB0, 0xA9, 0x06, 0xDA, 0x2D, 0x5B, 0xD7,
    0x8A, 0x66, 0x7B, 0xA5, 0xFF, 0x6A, 0xB9, 0xE0, 0xBE, 0x9E, 0x2C, 0x36, 0xF0, 0xD2, 0xD0, 0x97,
    0x26, 0x3E, 0xA2, 0x08, 0x5D, 0x58, 0x4F, 0x91, 0x70, 0x8B, 0xC0, 0x90, 0x0E, 0x00, 0x5A, 0xE7,
    0x45, 0x87, 0xB2, 0x83, 0x1C, 0xC4, 0x52, 0xD4, 0x96, 0xDE, 0x21, 0x04, 0x7A, 0x3F, 0x12, 0x30,
    0x67, 0x50, 0x09, 0x48, 0x42, 0xAB, 0xD6, 0xDD, 0x9B, 0xA4, 0x18, 0xE8, 0xC9, 0x4E, 0x46, 0x84
};

static byte[] Xor_Ord3_5 = {
    0xBF, 0xB7, 0x80, 0x05, 0x48, 0xDA, 0xF0, 0x77, 0xA0, 0x93, 0xE5, 0x0C, 0x07, 0x69, 0xE7, 0x2F,
    0xF4, 0x0D, 0x6F, 0xD4, 0x2C, 0x3B, 0x1E, 0x43, 0x71, 0xF5, 0x1A, 0xA8, 0x57, 0x31, 0x4B, 0x6D,
    0x35, 0x8E, 0xC8, 0x92, 0xED, 0x0F, 0xC2, 0xD7, 0xB9, 0x58, 0xB2, 0xC5, 0x7B, 0x18, 0x4A, 0x98,
    0x3C, 0x11, 0xBB, 0xFD, 0xAA, 0x41, 0xB5, 0x17, 0xE1, 0xF3, 0x22, 0x5B, 0xE4, 0x19, 0x9B, 0x42,
    0x81, 0xFF, 0x21, 0xA2, 0x64, 0xB1, 0x5E, 0x23, 0xC0, 0xDE, 0x28, 0xFC, 0x99, 0xF1, 0x0E, 0x9A,
    0x50, 0xB3, 0x09, 0x1B, 0xEC, 0x4D, 0x51, 0x6B, 0xD2, 0x33, 0x90, 0x79, 0x5D, 0x97, 0xB0, 0x60,
    0x1D, 0x63, 0xEF, 0xE9, 0x8F, 0x87, 0x75, 0x06, 0xBC, 0x68, 0xA5, 0x13, 0xD1, 0x26, 0x38, 0x82,
    0xBA, 0x04, 0x10, 0x56, 0xAF, 0x34, 0x62, 0x3E, 0x30, 0x5C, 0xAC, 0xE2, 0x91, 0x45, 0x2B, 0xCB,
    0xEE, 0x47, 0x2E, 0xB8, 0xA9, 0x96, 0xA3, 0x7C, 0xFE, 0xB6, 0xE0, 0xD3, 0xA6, 0xDF, 0x59, 0x84,
    0x32, 0xBD, 0xD5, 0xC6, 0x39, 0xA1, 0xA7, 0xBE, 0xAD, 0x4E, 0x66, 0x2D, 0xF8, 0x9E, 0xDC, 0xC7,
    0x7D, 0x03, 0x70, 0x40, 0x20, 0x8B, 0xE6, 0xD9, 0x7E, 0x85, 0xCC, 0x8A, 0x01, 0x16, 0xE8, 0x5A,
    0xAB, 0x4C, 0x74, 0x2A, 0x3D, 0xC9, 0x72, 0x29, 0xE3, 0x7F, 0x52, 0x94, 0x0A, 0x89, 0x8C, 0x37,
    0x1C, 0xEA, 0x3A, 0xD6, 0x6A, 0xB4, 0xC1, 0x65, 0x55, 0x3F, 0xF6, 0x08, 0x36, 0x95, 0x0B, 0x9F,
    0x73, 0xCD, 0x7A, 0x15, 0xCE, 0x9C, 0x14, 0xCF, 0x46, 0xFB, 0x02, 0xCA, 0xDB, 0x88, 0xF9, 0xC4,
    0x49, 0xEB, 0xDD, 0x6C, 0x00, 0x4F, 0x6E, 0xF2, 0x67, 0x24, 0xD0, 0x25, 0x9D, 0x54, 0xA4, 0xAE
};

Of interest is that the file is packaged the same as the newer DSLR firmware files, but that the A firmware is the larger file, while the B firmware is tiny.

I have no plan to start work on a J1/V1 firmware hack presently, as the D5100 work is all occupying, but people are welcome to come over to the Nikon Hacker forums, and discuss this work if they would like to progress it, or help with the DSLR work.

[Update] The A firmware does not appear to be a Fujitsu FR CPU like the DSLR are.

Also there are references to “SANYO” and “SANYO Digital Camera” in A firmware, how very strange Nikon.

[Update: 25th March - Thanks to Петр for pointing out that I had a row missing from table three due to cut’n’paste errors]

Interested in more, come join the us at Nikon Hacker, or use the Online Patch Tool (Help)

New Transformers are Crazy Hard

When I was ten, Transformers where not that hard to transform.

Meet modern Bumblebee,

Bumblebee
Bumblebee

This transformer was crazy hard to transform, in the couple of days I was visting with the 4 year old owner, the head “popped off” multiple times, and I couldn’t get it back on without feeling I was going to break the toy. Lucky said child’s mother had the magic touch.

But transforming challenge aside, it was a pretty cool model/toy. Just not like the Starscream or Optimus Prime of my days gone by….

Expectation Stress

Wow, since doing the easy work of decrypting the newer Nikon firmware, I’ve felt an immense pressure to pull another rabbit from the hat, when in reality the last reversing engineering project I worked on took years (2-5), and thus I feeling real burnt out. a) due to trying to find the next “cool” insight, and 2) wanting to be part of the action, and keep up with others are finding.  (the index scheme is an insiders joke)

To this end, I point would-be-helpers to nikonhacker.com, I’ve been contributing there, as best I can.

It’s very addictive having such large inflows of traffic to your site/blog, that I was reluctant to redirect it, but I have also felt since November a presure not to post stupid stuff like I used too, or am about to post, thus have been silent (besides the travel and working on things that are not up for chatter).

So there’s my Nikon update…. I have been having a good family holiday..

Monopoly Deal

Monopoly_Deal
Monopoly_Deal

The family was given Monopoly Deal for Christmas, and it’s a fantastic game.

My first game was a six person double deck game, which was slower, and very brutal. Since then I have played many a game with Jacob (single player) and it’s fast and fun.

In fine to twenty minutes, you can compact all the fun and rage of the full 4 hour game, but it’s done, and you can move on so much quicker. It has stealing, ripping people off, double crossing, saying no (it’s a card, but it the best feeling in the game rejecting a big play, just watch out for the reverse no,back at you…).

It’s a very fun game, and while there it can have all the rage of the original it, also can be started and finished within 30 minutes.. so many hands can be played giving the “balance” missing from Monopoly.

IDA Script: Fixing 16bit pushed data segment references

A good friend has started reversing an old 16bit Borland C++ (3.1?) program, and had lots of stack push data segment offsets that were not correctly cross referencing.

After telling him the shortcuts for manually fixing the issue (press O for the data segment, or Alt-R for any segment offset), he wrote an IDC script to do it en mass.

Thus (made up example code)

push ds;
mov ax, 0x1234;
push ax

should look like:

push ds;
mov ax, ds:dword_1234;
push ax

Here’s his script:

#include <idc.idc>

static main()
{
  auto seg, loc;
  auto movloc, movtarget;
  auto xref;
  auto dsegbase;

  dsegbase = SegByName("dseg") * 16;
  Message("dsegbase=%x\\n", dsegbase);

  Message("========================================\\n");
  seg = FirstSeg();

  while(seg != BADADDR )
  {
    Message("----------------------------------------\\n");

    loc = SegStart(seg);

    if( Byte(loc) != 0xCD || Byte(loc+1) != 0x3F)
    {
      Message("Fixing indirect push [ds:xx] refs from %s\\n", SegName(seg));

      while(loc != BADADDR && loc < SegEnd(seg))
      {
        if (GetMnem(loc) != "push" || GetOpnd(loc, 0) != "ds")
        {
          loc = NextHead(loc, BADADDR);
          continue;
        }
        loc = NextHead(loc, BADADDR);

        if (GetMnem(loc) != "mov" || GetOpType(loc, 1) != o_imm)
        {
          loc = NextHead(loc, BADADDR);
          continue;
        }
      movloc = loc;
      movtarget = GetOpnd(movloc, 0);
      loc = NextHead(loc, BADADDR);

      if (GetMnem(loc) != "push" || GetOpnd(loc, 0) != movtarget)
      {
        continue;
      }

      // At this point, we know we're pushing a [ds:x] combo.
      //Message("%x: mov %s, %s\\n", movloc, movtarget, GetOpnd(movloc, 1));

      // Abort if there already exists a Dxref
      xref = Dfirst(movloc);
      if (xref != BADADDR)
      {
        continue;
      }

      Message(" Updating %s:%04x\\n", SegName(seg), (movloc - seg) & 0xffff);
      OpOff(movloc, 1, dsegbase);
      }
    }

    seg = NextSeg(seg);
  }
}

Nikon Firmware Insights #05

Just to let people know, yes I’ve been working on understanding the D5100 firmware.

I’ve got most the area’s of code identified (where they are, not what they do), but there are some puzzles, with some chunks of code that are used (eg selects a picture to be shown based on shooting mode) but the code it self is not directly linked to, and it’s address in not present in the image. There is defiantly some form of jump/call table compression/encoding done, as there are functions that do some maths, and then call the result. So that needs to be puzzled out.

So to help map the data (and thus remove possible options from above puzzle), I previous mentioned mapping the jpg’s out:

Embedded Jpg's
Embedded Jpg's

As can be seen in this small sample, there’s the icons for the different shooting modes, and three colour schemes.

Last night I was working on using a modified version of BinViz (original found here) and have found how the font’s and overlay text/images are packed, and I am in the process of tracking down how the width/size information is encoded in the associated data tables. Shown below is the same block of data shown, but at two different widths, showing the “Dial” overlays and the “Bulb Time” text:

Overlay Images 2
Overlay Images 2
Overlay Images 1
Overlay Images 1

It’s quite neat looking at the Asian font sets, as the fonts/overlays use subpixel rendering, which can be seen in the green arm of the sports mode dial icon above. Much simpler, the normal overlays are just black/white, and now look better X/Y scaled.

I started a Google Code Project called Nikon Firmware Tools in which I’m placing the tools I’m using and the changes I’m making to them as I go. So interested developers can look there. Sorry only code so far.

Interested in more, come join the us at Nikon Hacker, or use the Online Patch Tool (Help)

Nikon Firmware Insights #04

A little under 24 hours ago, and roos posted he and Kungsholmens Kameraklubb found the checksum for the D7000 A & B firmware , but it didn’t work on the d5100 firmware.

The code they used was the big endian form of the CRC16, but running on little endian CPUs.

Turning the Wikipedia CRC code into C# follows:

For the D7000 A & B, D300S A&B, D3S A & B, D3100 A, D5100 A firmware files use start: 0x0000, mask=0x1021

For all models (D3S, D300S, D7000, D5100 & D3100) the CRC is a normal Xmodem CRC16, the originally reported difference for D5100 & D3100 was due to a XOR error (by me), that is now found & fixed.

static int crcBig(byte[] data, int len)
{
  int rem = 0x0;

  for (int i = 0; i < len; i++)
  {
    rem = rem ^ (data[i] << 8);
    for (int j = 0; j < 8; j++)
    {
      if ((rem & 0x8000) != 0)
      {
        rem = (rem << 1) ^ 0x1021;
      }
      else
      {
        rem = rem << 1;
      }
      rem = rem & 0xFFFF; // Trim remainder to 16 bits
    }
  }
  // A popular variant complements rem here
  return rem;
}

For the D5100 B firmware use start= 0x4ed4, mask= 0x1021

D5100 checksum passed, now loading firmware
D5100 checksum passed, now loading firmware
D5100 HaCkEd firmware running.
D5100 HaCkEd firmware running.

The D3100 B firmware also matches the same start value, so I’d assume it’s common to both. And the only thing I modified here was the firmware help message, and I’ve not tested if you can load the 1.01 firmware over top it’s self. That is the next step. Also to make a tool (Vitaliy?) the patches and encrypts for you to avoid typo’s.

[Update]
Very import and good news, firmware lets you flash same version over top it’s self. Thus my camera is now back running normal Nikon 1.01 firmware. Warranty unbroken!
[Update 2: 27 Nov] Fixed CRC code in light of XOR code change.

Interested in more, come join the us at Nikon Hacker, or use the Online Patch Tool (Help)