Debug, test & contribute

Turn an observation into a reproducible fix and a reviewed patch.

7 min readChapter 13 of 50Guided platform development
In this chapter

When a platform change builds but behaves unexpectedly, collect evidence before changing code. This chapter teaches a repeatable debugging loop, shows how to identify an actual Android 14 native test target, and explains what logs and Gerrit can and cannot establish. The following chapter performs a controlled, reversible fault injection in a real pinned utility and its existing test.

Before you start#

The diagnosis exercises require the Android 14 checkout. Device-based tests require a disposable matching Cuttlefish device; a source-only learner can still inspect declarations and reason through the expected failure without claiming a test ran.

Learning goals#

  • Record a reproducible symptom and verify source/image identity.
  • Treat logs, stack traces, and tombstones as evidence with limits.
  • Derive a real test module and test selector from pinned test source and Android.bp.
  • Preflight Repo branch operations before changing a shared workspace.
  • Distinguish local test/build evidence from upstream review and merge.

Practice#

Record a symptom and establish the layer

From symptom to tested repair
  1. Reproduce and identify

    Record expected/actual behavior, target build, source state, and minimal steps.

  2. Trace and predict

    Follow evidence to callers, implementation, module, and a relevant assertion.

  3. Test one proposed cause

    Establish a baseline; change one variable; preserve complete results and failures.

  4. Check repair and regressions

    Repeat the original case, a negative case, and relevant neighboring behavior.

Conceptual model. An investigation workflow, not proof of an executed test. Review/upload and upstream acceptance are additional outcomes, not implied by local success.

Write down expected behavior, actual behavior, exact steps, frequency, clean-boot state, device, and ro.build.fingerprint. A reproducer is the smallest sequence that reliably shows the issue. A stale image can make a correct source change look ineffective. Before editing, run repo status at the workspace root and git diff -- in the affected project. If you cannot account for existing changes, stop.

Classify the first evidence instead of immediately opening a random file:

  • UI/app crash: process and Java/Kotlin exception; inspect app logs and source.
  • Framework/service failure: identify the service process (often but not always system_server) and its caller/API path.
  • Native daemon crash: native process, signal, and tombstone/stack trace.
  • Kernel failure: kernel logs/oops and kernel-side evidence; userspace log silence is not proof.
  • Test/infrastructure failure: runner, device preparation, timeout, or transport diagnostics may fail before product behavior is exercised.

These are starting hypotheses, not a diagnostic verdict. A stack’s top frame is where failure surfaced, not necessarily the root cause. Logs may expose account/device/network identifiers; capture privately and redact before sharing.

Read bounded logs and native crash evidence

Confirm the device and capture a finite snapshot:

bash
adb devices -l
adb -s SERIAL logcat -d -t 500

Replace SERIAL with the intended device serial. Find the time window around the reproduction, then filter only after seeing relevant tags/text; broad filters can hide context. adb logcat -c clears diagnostic evidence, so do not run it before preserving what you need.

A native tombstone may contain signal, thread, registers, and stack frames. On a development build it is commonly under /data/tombstones/ (the native crash guide notes the tombstone is written there), but access and path vary. Do not root a device or change permissions just to retrieve it. Symbolize only with the exact binaries and symbols from the crashing build; mismatched symbols can misidentify functions. Follow the release-specific Android native debugging guidance rather than copying a command for a different build layout.

Discover and run a real Android 14 test

Do not infer an atest identifier from a filename. At android-14.0.0_r1, system/core/init/Android.bp declares the native cc_test module CtsInitTestCases, with source rlimit_parser_test.cpp. That file defines GTest case rlimit and test method RlimitSuccess; the Atest GTest selector format combines these as follows:

bash
source build/envsetup.sh
lunch aosp_cf_x86_64_phone-userdebug
adb devices -l
SERIAL='replace-with-the-intended-disposable-cuttlefish-serial'
atest -s "$SERIAL" CtsInitTestCases:rlimit#RlimitSuccess

This is a real tag-source-derived target and test selector, not a placeholder. The test module declares require_root: true, so use the matching disposable userdebug test device and replace SERIAL with its exact value from adb devices -l. Atest's -s/--serial option selects one device (Atest docs: "One device can be tested at a time"). Atest may build the test and prepare/run it on that device; it can take time. Do not infer success from command launch. For a broader check, atest -s "$SERIAL" CtsInitTestCases runs the module, which is larger than this selected method.

Output varies by Trade Federation and build. Evidence to retain is: the resolved test/module name, build/install/run stages, a per-test PASSED or FAILED result (or a clearly identified setup/infra error), and the command exit status. A build-only message or device-preparation failure is not a passing test. The atest selector syntax is documented for GTests as module:test-case#test-method; the pinned source supplies CtsInitTestCases, rlimit, and RlimitSuccess. No success transcript is printed here because this lesson has not executed the test.

If the target is not recognized, re-open the tag-local Android.bp, confirm the checkout tag, and run atest --help; do not substitute a filename, directory, or guessed module name. For other features, repeat this evidence chain: test source → TEST(case, method) or class/method → enclosing build declaration’s name → documented runner selector. Generated source is not the test’s source of truth; inspect its generator/interface and corresponding implementation/test.

Branch and review safety

Repo branches are project-specific. Before starting one, inspect state and existing names; these commands do not alter the checkout:

bash
repo status
git -C system/core status --short
git -C system/core branch --show-current
git -C system/core branch --list 'fix-small-issue-*'

Proceed only if the project path is correct, local changes are understood, and the branch name you choose is unused. Then use a unique name:

bash
repo start fix-small-issue-rlimit system/core
repo status
git -C system/core branch --show-current

If the project has unfamiliar edits or the branch state is unexpected, stop before repo start; ask the owner rather than switching, resetting, or overwriting it. Review only your intended project paths with git diff --check and git diff. Stage specific files, not git add -A.

Before an upstream submission, read AOSP CLA/header requirements. A signed-off commit (git commit -s) and repo upload are part of the review workflow, not proof of acceptance. Check the CLA and license header requirements first. Gerrit upload creates/updates a review; reviewers and presubmit still decide whether it is ready. Never claim a test passed if it was not run.

Knowledge check#

  1. What can a stack trace tell you, and what can it not prove?
  2. How is CtsInitTestCases:rlimit#RlimitSuccess derived?
  3. Why must the matching symbols come from the same image as a native crash?
  4. What is the safe action if repo status shows unfamiliar changes before branching?

Answers#

  1. It shows a path of calls and where a failure surfaced, but not necessarily its root cause or whether the observed path is the only cause.
  2. CtsInitTestCases is the cc_test name in the pinned Android.bp; rlimit and RlimitSuccess are the GTest case and method in rlimit_parser_test.cpp.
  3. Code layout and addresses vary between builds; symbols from another build can map the same address to the wrong source.
  4. Stop and identify/coordinate ownership. Do not start a branch, stage, or modify state you cannot explain.

Checkpoint#

Pass when: you can show a reproducible symptom record; explain which evidence identifies the failing layer; derive the actual test selector from its source and Android.bp; and demonstrate a preflight before any branch creation. For the device run, retain the selected serial, test result, and exit status. A failed test is useful evidence; distinguish product failure from setup/infra failure rather than relabeling it.

Revise: if you cannot derive the module or selector, return to source-to-module navigation and inspect the tagged build declaration and test macro together. If the runner stops before executing the selected case, revisit build and target readiness; retain the setup error instead of treating it as a test failure. If you cannot run the large test, complete the source-tracing portion and mark execution not run. Retry only after identifying the blocker. Next: AOSP investigation lab.

Sources#

A little further beneath the surface.

Check your understanding with the checkpoint. Marking a chapter as read saves a bookmark, not a proficiency assessment.