Guide to Full ROM backup/dump for MTK6573 & MTK6516 phones

Hi all MTK6573/MTK6516 users !!!!


SEE SECOND POST FOR MTK6516 phones.

THIS POST IS FOR MTK6573 phones.

This is for all MTK6573 users with Android 2.3(.x). Also for all B63M phones !!

Follow this guide to backup your FULL ROM !! You can even flash your phone through FlashTools with this COMPLETE backup !!!

ROOT IS REQUIRED !!


Here is the guide:

1) ROOT is necessary. It is recommended to put the phone in 'Airplane mode' so that it won't disturb the process and make sure you have ~500 mb space in SD card.

2) Download 'MTK-6573-BackUpTools.rar' from attachments and extract it to your SD Card. You'll have 2 folders - 'gscript' and 'Install'.

3) Install 'GScriptLite.apk' from gscript folder.

3) After installation, open the 'GScriptLite' application -> Menu -> Add Script.

4) Tick 'Needs SU ?'

5) Click 'Load File' and select 'Back23.sh' and then select 'save'

6) Now Click on 'Back 23' and Super User will ask for Permission and allow it. Nothing more to touch. Wait for few minutes. After few mins, it will say 'Auto Close is Cancelled'. Now its safe to close.

7) Your Back-Up will be in 'backup_' folder located in your SD card's root.
It will have files named ' firmware.info, preloader.img, nvram.img, seccnfg.img, uboot.img, boot.img, recovery.img, secstatic. img, misc.img, logo.img, expdb.img, cache.img, system.img, data.img' in that folder. Copy it to your computer and keep it at a safe place. More Over if you are posting in the forums or sharing it online, don't include ' data.img, cache.img and nvram.img' as they contain your personal information.


NOTE: The backup script is not created by me. Gathered all information from the web !!

Attached Files

MTK-6573-BakUpTool.rar -  (426.2 KB, 10060 views)

Creating a ROM dump of your MT65x3 device

Well, I have decided to create this tutorial for those that wish to create a backup of the stock ROM installed on their MT65x3 based smartphones. This is actually the only working method for non-rooted phones.

I assume that you are already familiar with SP Flash Tool, but if not, just read my MT65x3 flashing tutorial.

First of all, you have to know the MTD partition table of your device. If you already have ADB installed on your computer, launch a Command Prompt / Terminal window on your computer and enter the following commands:

adb shell
cat /proc/mtd

The result should be something like this (please note that it can be different on your phone):

Copy the output into a file named firmware.info or alternatively, simply run the following commands to create that file on your SD card:

adb shell
cat /proc/mtd > /sdcard/firmware.info

After that, use MT65xx ROM Studio (thanks to linerty) to load firmware.info and generate the corresponding scatter file that will be needed for SP Flash Tool.

Make sure that the latest MT65x3 USB VCOM drivers are installed, open SP Flash Tool and mark USB Mode under Options tab.

Load the previously generated scatter file (MT6573_Android_scatter.txt) and ignore any message that may appear.

It can be noticed that every partition has it's begin address... Using a hexadecimal calculator you can easily determine the size of each partition. For example, let's say that you wish to backup the boot image (BOOTIMG partition), taking into consideration that it begins on 0x00480000 and the next partition begins on 0x00A80000, then the partition size is 0x00600000.

After having determined the partition size, select the "Read back" tab and this screen should appear:

Now you should edit the entry, enter a name for the backup file and the details that were previously determined (start address and length). Be sure to select the correct read method: Read Page Only.

Close the window and press F10 (or hit Read back button - the third one with an arrow). Now, with your phone completely turned off (to make sure your phone is completely turned off, remove the battery and replace it after a few seconds), hold VOL-UP button and connect the USB data cable into your phone. The process will start as soon as you see the red progress bar.

Followed by the blue progress bar...

When finished, you should see the green circle.

There you go, you have successfully made a backup of your boot.img. If you wish to backup more NAND partitions, just add more entries to the read back list.

Attention: Please follow the instructions carefully. I will not take any responsibility on whatever may happen with your phone.

Active Password Resetter

Active password changer usage guide.
This is for those  who have forgotten  their windows xp passwords.
First download Active Password re setter here

1. 
   
   
   Boot the Windows operating system from “Active password changer” CD and chose the “Active Password changer” option.
2. 
   
   
   Chose the second option of “search for MS SAM Database on all hard disk and logical drives”.
3. 
   
   
   Press Enter to continue.
4. 
   
   
   Chose the account number for changing the password.
5. 
   
   
   Press “Y” to save the changes and exit.
6. 
   Press any key to restart the system.

Another guide on android kernel compilation

Hello everyone, want to compile an Android kernel on your own? Than why not begin doing it so by following the simple guide below?

1) Setup Environment:

First you will need these things,

• A Linux PC (32 bit will work for compilation of kernel)
• Basic knowledge of using terminal in Unix based operating system
• Few packages for compilation
• Tool chains for compilation
• Kernel source code
• Patience :p

So now we begin, First of all open terminal in your Linux PC. I am using here Linux Mint 16, a Debian based Linux Distribution. You can use any Linux distro for it such as Ubuntu, fedora, Arch Linux, etc. procedure will be same just you need to use different commands with terminal as per your distro. Now type this code in your terminal:

sudo apt-get install git-core gnupg flex bison gperf build-essential zip curl libc6-dev libncurses5-dev:i386 x11proto-core-dev libx11-dev:i386 libreadline6-dev:i386 libgl1-mesa-glx:i386 libgl1-mesa-dev g++-multilib mingw32 openjdk-6-jdk tofrodos python-markdown libxml2-utils xsltproc zlib1g-dev:i386 git

This will install required packages for kernel compilation. Now last thing, we need tool chains for compiling kernel. You can get it from here: https://github.com/crossfire77/Android_Toolchains Download it and extract it somewhere. Then, We have to setup path of our tool chains, so type this in terminal:

gedit .bashrc

It will open one document, then put this codes at the end of it.

export PATH=${PATH}:path/to/toolchain/folder/arm-eabi-4.4.3/bin

Now we are done with downloading and setting up things. Let’s move to next step!

2) Download Source Code:

Kernel source codes are under GNU GPL open source license so vendors (ex. Sony, HTC, Samsung, LG) have to release it. You can find your appropriate kernel source code from mentioned websites:

• For Sony: http://developer.sonymobile.com/
• 
  
• For Samsung: http://opensource.samsung.com/
• 
  
• For HTC: http://www.htcdev.com/
• 
  
• For LG: http://www.lg.com/global/support/opensource/index

Just download it from mentioned websites as per your device and extract it somewhere in your computer.

3) Getting Config file:

Now this is where you should put your focus. For building a kernel you need config file. There is two ways for it:

• You can either get it from your phone with help of ADB (Android Debug Bridge) or
• You can get it from your kernel source code.

I will show you both the ways of doing it so.
1) Getting config file from your phone with the help of ADB: 

• First of all connect your phone to your computer.
• Then type the mentioned code below:

adb pull proc/config.gz ~/home/crossfire77/kernel

This will pull your existing config file from your phone and put it in kernel folder. Now extract it,

zcat config.gz > .config

So we have directly config for building our kernel.
NOTE:  Many times users’ faces errors while connecting phone with computer via ADB, it’s mostly issue of a permission. There is two solutions for it.
1.) Using ADB as root user: $sudo -s $ adb kill-server $ adb start-server $ adb devices And you will see your device now in terminal, it is a temporary solution meaning you have to do every time you connect your phone.
2.) Solve permission problem permanently:
Open terminal and type this: $ sudo gedit /etc/udev/rules.d/51-android.rules and copy content from this file, http://goo.gl/bldQS9 In this file, please replace “crossfire77” with your username.
Now last thing to do is, giving executing permission to file, and we are good to go.

$ sudo chmod a+r /etc/udev/rules.d/51-android.rules $ sudo service udev restart

This will solve your problem with ADB and FASTBOOT while connecting through them your phone with computer.  
2) Getting config file from kernel source code:
This part is little tricky. For this you have to have little knowledge about your device chipset number. In kernel source code there are lots of config files so from your chipset number you can find your config file. As example, My device is HTC Wildfire S and it has MSM7227 chipset so I can find its config in mentioned path: arch/arm/configs/msm7227_defconfig
So now just type this code in terminal:

make ARCH=arm CROSS_COMPILE=arm-eabi- msm7227_defconfig

It will create .config file from your existing chipset config and then we are good to go. As I told you before you need to have little knowledge about your device chipset then you can able to tell what is your correct config file, so be careful while doing this step. Everything is done now so shall we start building?
4) Building:
Type this in your terminal,

make ARCH=arm CROSS_COMPILE=arm-eabi- -jX

It is main command for compilation and here X can be replaced by maximum numbers of jobs your computer can handle simultaneously. For ease just enter the twice number of your cores. Ex. If your computer has two cores enter four.
NOTE: Please do not enter high number or your computer will explode due to overheating.
It will take around ~15-1 hour as per your computer hardware specs.so keep sit down and keep patience. After completing this procedure we will have our zImage (kernel)! Let’s create one working directory:

mkdir kernell

Now, we will copy our zImage to our working directory:

cp arch/arm/boot/zImage kernell

Next step, modules (modules builds with kernel and we need it for Wi-Fi and etc. as per your device requirement)

find . -name '*ko' -exec cp '{}' kernell \;

So we are done with the main part, now last step to make flash able zip out of this to flash our kernel.
5) Making Flashable Zip: We are done with compiling and building task and at the end we have our zImage (kernel) and modules. Now time to flash it to your phone, for it we have one very easy method, “any kernel updater” by koush. So download this file from here: http://goo.gl/mQpHhC
So, this was all about compiling an Android Kernel. Share me your any doubts or acknowledgements if you have.
Have fun compiling Android Kernel.

Android :The init process and init.rc

In this post I will wrap up the analysis of what is going on at boot by looking at the Android init process and the init.rc file. If we start with the source you will find the code for the init process under /system/core/init in the source tree. The init process is what will set up all native services and this is similar to a regular Linux system boot. At this stage we are not executing in the actual android runtime. There is however some Android specifics going on already at this level. One example of this is that the init.rc script is written in a specific format called the Android init language. This is covered in the readme file that is kept in the same directory as the init source. Since I have not seen very much about it on the net though I will post some of the main concepts here. I will also give a small example on how to add your own native service at startup.

The Android init language
The language used by the init process has four classes of statements. These are actions, commands, services and options. Actions are associated to commands in the way that an action is a sequence of commands. An action also need something called a trigger in that the sequence of commands will be executed when the trigger event has occured. An exemple may look like:

on boot
ifup lo
hostname localhost
domainname localdomain

This action will run a command sequence to set up basic networking once it receives the boot trigger. The supported commands include export, chmod, chown and a few others. The complete list of available commands can be found in the readme file.
Services are programs that are launched by init. They are associated with options which control how they are started. The syntax for starting a service is:
service < name > < pathname > [< argument > ]*
< option >
< option >
...
There is a number of options available and a few of them are:
critical - this is a critical process and init will restart the device if it exists more than four times in four minutes
user < username > - change to this username before executing the service
group < groupname > [< groupname > ]* - change to groupname before executing the service
setenv < name > < value > - set environment variable < name > to < value > in the launched process
The init script is also the place where some important system properties are set. This is done with the setprop command and include things like the memory thresholds for applications, tcp buffer sizes etc.
The init.rc file itself is found in the root directory on a device and in the source tree it is kept under system/core/rootfs. The below example is the actual init.rc file that I have shortened significantly. The basic concepts are clearly illustrated. First there is an action on init where a long sequence of commands is run. As you can see it is accepted to use comments and they are preceded by #. A number of variables are setup during init and file systems are mounted. The next step is the boot action where networking is set up and properties assigned to the system. Then there is a list of services to be started by init.

on init
sysclktz 0
loglevel 3
# setup the global environment
export PATH /sbin:/system/sbin:/system/bin:/system/xbin
export LD_LIBRARY_PATH /system/lib
export ANDROID_BOOTLOGO 1
export ANDROID_ROOT /system
...
# mount mtd partitions
1. 
Mount /system rw first to give the filesystem a chance to save a checkpoint
mount yaffs2 mtd@system /system
mount yaffs2 mtd@system /system ro remount
2. 
We chown/chmod /data again so because mount is run as root + defaults
mount yaffs2 mtd@userdata /data nosuid nodev
chown system system /data
chmod 0771 /data
...
on boot
# basic network init
ifup lo
hostname localhost
domainname localdomain
# set RLIMIT_NICE to allow priorities from 19 to -20
setrlimit 13 40 40
# Define the oom_adj values for the classes of processes that can be
# killed by the kernel. These are used in ActivityManagerService.
setprop ro.FOREGROUND_APP_ADJ 0
setprop ro.VISIBLE_APP_ADJ 1
setprop ro.SECONDARY_SERVER_ADJ 2
setprop ro.BACKUP_APP_ADJ 2
setprop ro.HOME_APP_ADJ 4
setprop ro.HIDDEN_APP_MIN_ADJ 7
setprop ro.CONTENT_PROVIDER_ADJ 14
setprop ro.EMPTY_APP_ADJ 15
...
## Daemon processes to be run by init.
##
service console /system/bin/sh
console
# adbd is controlled by the persist.service.adb.enable system property
service adbd /sbin/adbd
disabled

Starting your own service at boot.
Normally there is not much need to do things in the init script and start services of your own. What is provided in the system goes a long way and often it may be better to add something to the Android runtime instead of using the native services. If you have a reason to add something or just what to try things out however it is relatively straight forward to add a native service to that init sequence. I have created a simple example that writes the time elapsed since it started to the log. It wakes up every three seconds. The code for exampleservice.c looks like:

#define LOG_TAG "Example Service"
#include < utils/log.h >
#include < unistd.h >
int main(int argc, char **argv)
{
    LOGI("Service started");
    int elapsed = 0;
    while(1)
    {
         sleep(3);
         elapsed += 3;
         LOGI("Service elapsed time is %d", elapsed);
    }
}

Place this somewhere in the platform build system and build as an executable. The build system will place the output in the /system/bin folder on the device. Now all that is needed is to add the following to the init.rc script:

service exampleservice /system/bin/exampleservice
user exampleservice
group exampleservice
oneshot

and your service will start on init. It is also possible to write your own file using the Android init language that enables your services and variables. Then just add a line to import the file in init.rc and you are done.

The Android boot process from power on

 COURTESY OF http://www.androidenea.com

Since mobile platforms and embedded systems has some differences compared to Desktop systems in how they initially start up and boot this post will discuss the initial boot stages of an Android phone in some detail. Since we have used the Beagle Board as reference in some previous examples any specifics here are related to a similar system.

1. Power on and boot ROM code execution
At power on the CPU will be in a state where no initializations have been done. Internal clocks are not set up and the only memory available is the internal RAM. When power supplies are stable the execution will start with the Boot ROM code. This is a small piece of code that is hardwired in the CPU ASIC. For more information on boot ROM and configurations study the initalization chapter in
the Omap 3530 TRM.

• A. The Boot ROM code will detect the boot media using a system register that maps to some physical balls on the asic. This is to determine where to find the first stage of the boot loader.
  
• B. Once the boot media sequence is established the boot ROM will try to load the first stage boot loader to internal RAM. Once the boot loader is in place the boot ROM code will perform a jump and execution continues in the boot loader.
  

2. The boot loader
The boot loader is a special program separate from the Linux kernel that is used to set up initial memories and load the kernel to RAM. On desktop systems the boot loaders are programs like GRUB and in embedded Linux uBoot is often the boot loader of choice. Device manufacturers often use their own proprietary boot loaders. The requirements on a boot loader for Linux running on an ARM system can be found in the Booting document under /Documentation/arm in the kernel source tree.

• A. The first boot loader stage will detect and set up external RAM.
• B. Once external RAM is available and the system is ready the to run something more significant the first stage will load the main boot loader and place it in external RAM.
• C. The second stage of the boot loader is the first major program that will run. This may contain code to set up file systems, additional memory, network support and other things. On a mobile phone it may also be responsible for loading code for the modem CPU and setting up low level memory protections and security options.
• D. Once the boot loader is done with any special tasks it will look for a Linux kernel to boot. It will load this from the boot media (or some other source depending on system configuration) and place it in the RAM. It will also place some boot parameters in memory for the kernel to read when it starts up.
• E. Once the boot loader is done it will perform a jump to the Linux kernel, usually some decompression routine, and the kernel assumes system responsibility.

3. The Linux kernel
The Linux kernel starts up in a similar way on Android as on other systems. It will set up everything that is needed for the system to run. Initialize interrupt controllers, set up memory protections, caches and scheduling.

• A. Once the memory management units and caches have been initialized the system will be able to use virtual memory and launch user space processes.
• B. The kernel will look in the root file system for the init process (found under system/core/init in the Android open source tree) and launch it as the initial user space process.
  

4. The init process
The init process is the "grandmother" of all system processes. Every other process in the system will be launched from this process or one of its descendants.

• A. The init process in Android will look for a file called init.rc. This is a script that describes the system services, file system and other parameters that need to be set up. The init.rc script is placed in system/core/rootdir in the Android open source project.
• B. The init process will parse the init script and launch the system service processes.

5. Zygote and Dalvik
The Zygote is launched by the init process and will basically just start executing and and initialize the Dalvik VM.
6. The system server
The system server is the first java component to run in the system. It will start all the Android services such as telephony manager and bluetooth. Start up of each service is currently written directly into the run method of the system server. The system server source can be found in the file frameworks/base/services/java/com/android/server/SystemServer.java in the open source project.


7. Boot completed
Added this part to the post on 20090831 since it is very useful and something I should not have left out from the beginning. Once the System Server is up and running and the system boot has completed there is a standard broadcast action called ACTION_BOOT_COMPLETED. To start your own service, register an alarm or otherwise make your application perform some action after boot you should register to receive this broadcast intent.

The separate boot steps and possible places to add your own functionality are covered in more detail in separate posts.

INCREASE YOUR LINUX WEB SPEED !

Internet speeds in Ubuntu can be increased. Simply follow the steps.

• Open a Terminal via Applications->Accessories->Terminal  and type the following

                          sudo vim  /etc/sysctl.conf         (press i for edit mode)

• Then Paste the Following at the end of the file:

         ## increase TCP max buffer size setable using setsockopt()            
         net.core.rmem_max = 16777216                                          
         net.core.wmem_max = 16777216                                          
         ## increase Linux autotuning TCP buffer limits                        
         ## min, default, and max number of bytes to use                       
         ## set max to at least 4MB, or higher if you use very high BDP paths  
         net.ipv4.tcp_rmem = 4096 87380 16777216                               
         net.ipv4.tcp_wmem = 4096 65536 16777216                               
         ## don't cache ssthresh from previous connection                      
         net.ipv4.tcp_no_metrics_save = 1                                      
         net.ipv4.tcp_moderate_rcvbuf = 1                                      
         ## recommended to increase this for 1000 BT or higher                 
         net.core.netdev_max_backlog = 2500                                    
         ## for 10 GigE, use this, uncomment below                             
         ## net.core.netdev_max_backlog = 30000                                
         ## Turn off timestamps if you're on a gigabit or very busy network    
         ## Having it off is one less thing the IP stack needs to work on      
         ## net.ipv4.tcp_timestamps = 0                                        
         ## disable tcp selective acknowledgements.                            
         net.ipv4.tcp_sack = 0                                                 
         ##enable window scaling                                               
         net.ipv4.tcp_window_scaling = 1                                                    

• Then  type the follwing to exit and save what you have just done. Press ESC to quit the edit mode and type the following.

                                    :wq                         

• Then type the following to to apply the settings.

             sudo sysctl -p          

• You can disable all these settings by removing these lines you added via:

             sudo gedit /etc/sysctl.conf

Compiling and running your own Android kernel e.g on the Nexus 7 2013

Now at last we're getting to the interesting part: compiling and running our own modified kernel on the Nexus!

We are of course going to pick most of our information from the official kernel building guide, though, unlike the guide, we're going to go all the way through, and not stop before we are actually running our recompiled kernel.

This time, we'll use Linux as our development platform (more convenient). The first thing I need to warn you about then is, if you're running a pure 64 bit distro (such as Slackware x64), you're going to find that you must install a 32-bit compatibility layer to run the 32 bit debug utilities. This is a bit annoying, and I'm not going to guide you through that, so make sure you sort your platform issues, okay?

Installing the toolchain

Anyway, first order of the day is to get the official arm toolchain, which I'm going to install in /usr/local/share/ since I plan to keep using it for some time:

# cd /usr/local/share/ # git clone https://android.googlesource.com/platform/prebuilts/gcc/linux-x86/arm/arm-eabi-4.6 Cloning into 'arm-eabi-4.6'... remote: Sending approximately 124.64 MiB ... remote: Counting objects: 33, done remote: Finding sources: 100% (33/33) remote: Total 580 (delta 146), reused 580 (delta 146) Receiving objects: 100% (580/580), 124.64 MiB | 715 KiB/s, done. Resolving deltas: 100% (146/146), done.

Now, let's add that arm toolchain to our path:

# export PATH=$PATH:/usr/local/share/arm-eabi-4.6/bin # arm-eabi-gcc --version arm-eabi-gcc (GCC) 4.6.x-google 20120106 (prerelease) Copyright (C) 2011 Free Software Foundation, Inc. This is free software; see the source for copying conditions.  There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Ideally, of course, you'd want to add this export into your .profile or something so that you don't have to set it up every time.

Dude, where's the kernel source?

OK, now we want to fetch the kernel source of course, so let's do just that. The one thing you need to know is that the platform name for the Nexus 7 2013 is msm, thus:

# mkdir /usr/src/android # cd /usr/src/android/ # git clone https://android.googlesource.com/kernel/msm.git kernel Cloning into 'kernel'... remote: Sending approximately 850.89 MiB ... remote: Counting objects: 50642, done remote: Finding sources: 100% (5223/5223) remote: Getting sizes: 100% (699/699) remote: Compressing objects:  99% (11626/11627) remote: Total 3195022 (delta 2625647), reused 3194640 (delta 2625583) Receiving objects: 100% (3195022/3195022), 850.15 MiB | 716 KiB/s, done. Resolving deltas: 100% (2634067/2634067), done.

As can be seen from the above, expect a download of about 850 MB.

# cd kernel # ls #

Huh, there's nothing in there?

# git show commit f688a7654b339885e689d0f95f38b9daa3d85c0f Author: Jean-Baptiste Queru <jbq@google.com> Date:   Thu Nov 17 12:44:41 2011 -0800       empty commit lines 1-5/5 (END) # git branch -a * master   remotes/origin/HEAD -> origin/master   remotes/origin/android-msm-2.6.35   remotes/origin/android-msm-3.9-usb-and-mmc-hacks   remotes/origin/android-msm-flo-3.4-jb-mr2   remotes/origin/android-msm-mako-3.4-jb-mr1   remotes/origin/android-msm-mako-3.4-jb-mr1-fr   remotes/origin/android-msm-mako-3.4-jb-mr1-kgsl   remotes/origin/android-msm-mako-3.4-jb-mr1.1   remotes/origin/android-msm-mako-3.4-jb-mr2   remotes/origin/android-msm-sony-cm-jb-3.0   remotes/origin/master

Mmmm, OK, maybe the flo branch then?

# git checkout android-msm-flo-3.4-jb-mr2 Checking out files: 100% (41678/41678), done. Branch android-msm-flo-3.4-jb-mr2 set up to track remote branch android-msm-flo-3.4-jb-mr2 from origin. Switched to a new branch 'android-msm-flo-3.4-jb-mr2' root@stella:/usr/src/android/kernel# git show commit 9e52a2195889f4f03ddfcaefb19c0b42ec1d9070 Author: Praveen Chavan <pchavan@codeaurora.org> Date:   Fri Mar 29 17:28:31 2013 -0700       qseecom: Fix issue with incomplete command exiting prematurely       A process waiting on a signal can be awaken by any signal. We     need to only continue processing when the condition of the     wait event is met.       Change-Id: Ib2102babbb505876f89b04399729e6ff5a475605     Signed-off-by: Mona Hossain <mhossain@codeaurora.org>     Signed-off-by: Praveen Chavan <pchavan codeaurora.org>

Let me spare you the suspense there: As "documented" on their page, Google don't actually use branches or even tags for their development. Instead they force you to use an sha1 reference that's not attached to any helpful entity, in order to figure out the branch you should pick. Moreover, what they advise you to do to get that SHA is go for another large download of pointless binary data (650 MB) containing all the revision of their kernel files. Well, not everybody's running on Google Fibre... yet, and my ISP also enforces quotas, so, since we're smarter than this, we're going to avoid downloading 650 MB in exchange of a few SHA bytes. Instead, we'll head directly to https://android.googlesource.com/device/asus/flo-kernel and look at the most recent "flo: prebuilt kernel", which, at the time of this post, is:

79816e3  flo: prebuilt kernel by Jean-Baptiste Queru - 2 months ago jb-mr2-dev master tools_r22.2 android-4.3_r2.1

If you click on the link, you'll see part of the SHA we are really after here:

365a6e0 gpu: ion: Minimize allocation fallback delay

That 365a6e0 is the ID we're seeking, so let's use that (and also reuse the official kernel name, 'android-4.3_r2.1', for our branch name):

# git checkout -b android-4.3_r2.1 365a6e0 Switched to a new branch 'android-4.3_r2.1' # ls AndroidKernel.mk  CREDITS         Kbuild   MAINTAINERS  README          arch/   crypto/   firmware/  include/  ipc/     lib/                mm/   samples/  security/  tools/  virt/ COPYING           Documentation/  Kconfig  Makefile     REPORTING-BUGS  block/  drivers/  fs/        init/     kernel/  make_defconfig.sh*  net/  scripts/  sound/     usr/

That's more like it!

NB: We could be even smarter by just looking at the kernel revision provided by our device (3.4.0-g365a6e0), drop the whole part before the g and recognize that this is the partial SHA1 we are after. Then again, since there's a chance a new kernel was out and you're running an old or custom one...

Compiling the Android kernel

Now we need to set the variables that will tell the OS that we're cross compiling (again, something that you may want to do in your .profile using something like alias cross='export ARCH="arm";export SUBARCH="arm";export CROSS_COMPILE="arm-eabi-"'):

# export ARCH=arm # export SUBARCH=arm # export CROSS_COMPILE=arm-eabi-

Finally, we can initialize our default config and get going:

# make flo_defconfig   HOSTCC  scripts/basic/fixdep   HOSTCC  scripts/kconfig/conf.o   SHIPPED scripts/kconfig/zconf.tab.c   SHIPPED scripts/kconfig/zconf.lex.c   SHIPPED scripts/kconfig/zconf.hash.c   HOSTCC  scripts/kconfig/zconf.tab.o   HOSTLD  scripts/kconfig/conf warning: (ARCH_MSM_KRAITMP && ARCH_MSM_CORTEX_A5) selects HAVE_HW_BRKPT_RESERVED_RW_ACCESS which has unmet direct dependencies (HAVE_HW_BREAKPOINT) warning: (ARCH_MSM_KRAITMP && ARCH_MSM_CORTEX_A5) selects HAVE_HW_BRKPT_RESERVED_RW_ACCESS which has unmet direct dependencies (HAVE_HW_BREAKPOINT) # make menuconfig   HOSTCC  scripts/kconfig/lxdialog/checklist.o   HOSTCC  scripts/kconfig/lxdialog/inputbox.o   HOSTCC  scripts/kconfig/lxdialog/menubox.o   HOSTCC  scripts/kconfig/lxdialog/textbox.o   HOSTCC  scripts/kconfig/lxdialog/util.o   HOSTCC  scripts/kconfig/lxdialog/yesno.o   HOSTCC  scripts/kconfig/mconf.o   HOSTLD  scripts/kconfig/mconf scripts/kconfig/mconf Kconfig warning: (ARCH_MSM_KRAITMP && ARCH_MSM_CORTEX_A5) selects HAVE_HW_BRKPT_RESERVED_RW_ACCESS which has unmet direct dependencies (HAVE_HW_BREAKPOINT) warning: (ARCH_MSM_KRAITMP && ARCH_MSM_CORTEX_A5) selects HAVE_HW_BRKPT_RESERVED_RW_ACCESS which has unmet direct dependencies (HAVE_HW_BREAKPOINT)

Now that we're in the kernel config, let's start with something real basic, and change General setup ---> Local version - append to kernel release setting it to something like "hello". Hopefully, we'll be able to boot our recompiled kernel and see that string appear in the system info, to confirm that we are indeed using our own kernel. Off we go with building the whole thing then:

# make -j8 (...)   OBJCOPY arch/arm/boot/zImage   Kernel: arch/arm/boot/zImage is ready

We have a kernel — now what?

Aaaaand, that's pretty much where the Google guide stops. What do you mean, you also want to run your newly compiled kernel on your device?

If that's the case, then you should download more than 16 GB worth of build files from the infamous aosp (Android Open Source Project) suite —I wish I was kidding, see below— and then figure out how to inject your kernel in there. Good luck!

My usage data (in MB) after downloading the full aosp.
(Careful analysis may be required to notice the small difference in data usage)

The 16 GB of the full aosp (I was already 2 GB in for that day, hence the 18 GB above) sure put me dangerously close of getting a not so friendly call from my ISP for going over the monthly quota (again, not everybody is running on Google Fibre), so I might try to spare you this trouble.

If you're not going to download the aosp to try to figure out this mess, the first thing you should know then is that, as opposed to what is the case on most devices, testing your kernel on Android isn't as simple as pointing your bootloader to it. It must be included into a custom boot image (boot.img) along with a RAM disk and some other stuff.

Crafting an Android boot.img

The format of this Android boot images can be found in the bootimg.h of the mkbootimg tool from the android platform/system/core source. I guess we have to start somewhere, and we'll need to produce our own images, so let's start by fetching and recompiling mkbootimg then.

NB: Since I duplicated and improved on the original tools (see below), you can, and probably should, use https://github.com/pbatard/bootimg-tools.git instead of https://android.googlesource.com/platform/system/core for the git clone URL below.

# git clone https://android.googlesource.com/platform/system/core bootimg-tools Cloning into 'bootimg-tools'... remote: Counting objects: 92, done remote: Finding sources: 100% (92/92) remote: Total 19302 (delta 11674), reused 19302 (delta 11674) Receiving objects: 100% (19302/19302), 5.87 MiB | 655 KiB/s, done. Resolving deltas: 100% (11674/11674), done. # cd bootimg-tools/libmincrypt/ # gcc -c *.c -I../include # ar rcs libmincrypt.a *.o # cd ../mkbootimg # gcc mkbootimg.c -o mkbootimg -I../include ../libmincrypt/libmincrypt.a mkbootimg.c: In function 'main': mkbootimg.c:245:9: warning: assignment discards 'const' qualifier from pointer target type [enabled by default] # cp mkbootimg /usr/local/bin/ # cd ../cpio # gcc mkbootfs.c  -o mkbootfs -I../include # cp mkbootfs /usr/local/bin/

With the above, we'll be able to invoke mkbootimg and optionally mkbootfs to create an image. If you follow this guide closely, you'll see that we're not even actually going to use mkbootfs, but I kept it there just in case.

Now, we don't want to recreate our RAM disk from scratch, or have to guess all the parameters we'll need to use to create one, so let's pick a working one. For this you have two choices:

• Use a backup utility on your rooted Android device, and save the partition that contains the boot image. For instance, if you installed TWRP when rooting your device, and boot into recovery, then you will be able to save the 16MB Boot partition, which is what we are after.
• Download the latest factory image (at the time of this writing, it's JSS15R, 359MB big), extract it in full (tar then zip), and then pick the boot.img image from the zip (in JSS15R, that would be boot.img within image-razor-jss15r.zip). You also want to be mindful that bootloader-flo-flo-03.14.img is the bootloader image, and NOT what we are after.

Also, don't let the fact that in one case you get a 16MB file and in the other one that is less than 7MB worry you. The 16MB one is just padded with zeroes, which won't matter when we access its content.
For the remainder of this guide, even as it requires a 359 MB download, I'll go with using boot.img from the factory image.

Next thing we need is a way to unpack that boot.img file. There are a couple of unpack-bootimg.pl/split-bootimg.pl perl scripts dating back from 2008, so we can pick one of those and then use a binary editor take a look at the .img and extract the addresses and parameters we should provide mkbootimg, and try to cross-reference those with... Aaaaagh, I can't stand it any more!!!

That's "Aaaaagh!" with five A's

What the £$%^& is wrong with Android developers?!? Why the heck aren't these unpacker scripts providing you with the full set of parameters you need to use to repack an image, instead of having to look at a binary header, or some mysterious settings file in a 16GB repository to find them? Also, why, in 5 years, hasn't anyone, including Google, improved these tools and scripts? The thing is, even mkbootimg has issues and won't work on Windows (when compiled with MinGW) due to the use of non-Windows friendly open(), read(), write() et cetera.

That does it!

I'm gonna write my own unpack tool in C, fix mkbootimg in the process, and put it all on github.
Sheesh, as if I didn't have better things to do... I hope you're happy!

So off we go to the directory you compiled mkbootimg (NB: I you cloned from https://github.com/pbatard/bootimg-tools.git earlier, you can skip the wget line):

# cd /usr/src/android/bootimg-tools/mkbootimg/ # wget https://raw.github.com/pbatard/bootimg-tools/master/mkbootimg/unmkbootimg.c # gcc -o unmkbootimg unmkbootimg.c # cp unmkbootimg /usr/local/bin/

Now at last, you have the minimum of unmkbootimg, mkbootimg and mkbootfs installed in your path, and we can get going with our testing of the kernel.
I'll assume that you have a /usr/src/android/boot where you copied your boot.img, so let's get cracking:

# cd /usr/src/android/boot/ # ls boot.img # unmkbootimg -i boot.img kernel written to 'kernel' (6640200 bytes) ramdisk written to 'ramdisk.cpio.gz' (399979 bytes)   To rebuild this boot image, you can use the command:   mkbootimg --base 0 --pagesize 2048 --kernel_offset 0x80208000 --ramdisk_offset 0x82200000 --second_offset 0x81100000 --tags_offset 0x80200100 --cmdline 'console=ttyHSL0,115200,n8 androidboot.hardware=flo user_debug=31 msm_rtb.filter=0x3F ehci-hcd.park=3' --kernel kernel --ramdisk ramdisk.cpio.gz -o boot.img # ls boot.img  kernel  ramdisk.cpio.gz

As an aside that you don't need to run, but since it should be elementary that this is the basic functionality you want from a proper boot image unpack tool, we can confirm that the data provided by the unpack tool will produce a boot.img that is binary identical to the original one:

# mkbootimg --base 0 --pagesize 2048 --kernel_offset 0x80208000 --ramdisk_offset 0x82200000 --second_offset 0x81100000 --tags_offset 0x80200100 --cmdline 'console=ttyHSL0,115200,n8 androidboot.hardware=flo user_debug=31 msm_rtb.filter=0x3F ehci-hcd.park=3' --kernel kernel --ramdisk ramdisk.cpio.gz -o myboot.img # ls boot.img  kernel  myboot.img  ramdisk.cpio.gz # cmp -l boot.img myboot.img #

Now, THIS is what you want to be able to do from an Android image unpacking tool. Is it really that much to ask?

Crafting an Android boot.img 2: 'Electric boogaloo'

Moving on. Since we just want to test a kernel, we shouldn't really have to touch the cpio image (ramdisk), but then again, my goal here is to give you as many pointers as I can, so we might as well see how we craft our own ramdisk while we're at it. What we're going to do here, as an academic exercise, is add an it_works file at the root of the filesystem, which we'll look for after we booted, to confirm that we are able to use our modified stuff all the way through:

# mkdir ramdisk # cd ramdisk # gunzip -c ../ramdisk.cpio.gz | cpio -iu # ls charger*       init*             proc/              sys/ data/          init.flo.rc*      property_contexts  system/ default.prop   init.flo.usb.rc*  res/               ueventd.flo.rc dev/           init.rc*          sbin/              ueventd.rc file_contexts  init.trace.rc*    seapp_contexts fstab.flo      init.usb.rc*      sepolicy # touch it_works # ls charger*       init*             it_works           sepolicy data/          init.flo.rc*      proc/              sys/ default.prop   init.flo.usb.rc*  property_contexts  system/ dev/           init.rc*          res/               ueventd.flo.rc file_contexts  init.trace.rc*    sbin/              ueventd.rc fstab.flo      init.usb.rc*      seapp_contexts # find . | cpio -o -H newc | gzip > ../myramdisk.cpio.gz 1410 blocks # cd .. # ls boot.img  kernel  myramdisk.cpio.gz  ramdisk/  ramdisk.cpio.gz

We're finally set for the last part, where we copy the kernel we compiled earlier, and invoke mkbootimg with the set of parameters we got from unmkbootimg, and use both our modified kernel and cpio image:

# cp /usr/src/android/kernel/arch/arm/boot/zImage . # ls boot.img  kernel  myramdisk.cpio.gz  ramdisk/  ramdisk.cpio.gz  zImage* # mkbootimg --base 0 --pagesize 2048 --kernel_offset 0x80208000 --ramdisk_offset 0x82200000 --second_offset 0x81100000 --tags_offset 0x80200100 --cmdline 'console=ttyHSL0,115200,n8 androidboot.hardware=flo user_debug=31 msm_rtb.filter=0x3F ehci-hcd.park=3' --kernel zImage --ramdisk myramdisk.cpio.gz -o myboot.img # ls boot.img  kernel  myboot.img  myramdisk.cpio.gz  ramdisk/  ramdisk.cpio.gz  zImage*

Finally, a custom boot.img we can test. Let's press on by copying this myboot.img file into the directory we have adb and fastboot installed (see our earlier post about rooting the Nexus 7) and run the following set of commands which, unlike what many other guides seem to advise (what the heck, guys?), is NOT going to flash the kernel/boot.img but simply run it from memory. This means that, in case there's any mishap, you can simply power the Nexus off and you'll be good as new:

# ./adb start-server * daemon not running. starting it now on port 5037 * * daemon started successfully * # ./adb reboot bootloader # ./fastboot boot myboot.img downloading 'boot.img'... OKAY [  0.223s] booting... OKAY [  0.023s] finished. total time: 0.246s

The moment of truth!

After a long few seconds of anxiety, where the device doesn't seem to do anything, you should see either one of these two things:

• The Nexus' multicoloured 'X', telling you that your boot.img file was fine and that the tablet is now booting it, or
• The Google logo with the open lock, indicating that the device didn't accept your boot image and is restarting with the one it has in flash (or complete freezout, if which case you should press the power button until your device turns off)

If you've seen the multicoloured 'X' straight away, then there's a good chance that, if you go to your tablet settings, you'll see the following reported:

A few things you should recognize above:

• That hello appended to the 3.4.0 kernel version, that's the "local version" string we chose to add to our kernel when we recompiled
• root@stella as the user who compiled this kernel (yeah, I tend to compile stuff as root - so sue me!)
• That 365a6e0 after the g (for 'git') in the version number, that's the SHA1 from the git commit we had to pick to compile our kernel.

All in all, this looks fairly good. And a quick look to the root filesystem (through adb shell or through ssh) will also confirm that our it_works file is there, so we can also add whatever we want on the initial filesystem. Neat!

From there on, you can go ahead and tweak your kernel and initial filesystem exactly as you see fit. And after you have tested that everything works as it should, you can go ahead and flash the boot partition with your shiny custom boot.img, using:

fastboot flash boot boot.img

For the record, if you actually modify the source of your kernel, you should see a -dirty qualifier appended to your kernel version, which is normal, and won't prevent you to run a modified kernel.

Now what are you doing still reading this guide: get hacking your device already!