Flash storage qualification plan: samples to production

A vendor sends two samples. They enumerate, copy a test file quickly and survive a day on the bench. Purchasing marks the part “approved.” That is sample evaluation, not production qualification.

A qualification plan connects business risk to measurable limits. It defines exactly what was tested, in which host, under what workload and environment, and what evidence allows production release.

Start with the product requirement, not a benchmark

Write application requirements before choosing tools:

• interface, capacity and mechanical envelope;
• operating and storage temperature;
• startup and enumeration time;
• sustained read/write throughput after cache effects;
• average and tail latency limits;
• endurance and expected host-write profile;
• retention and service interval;
• power-loss and unsafe-shutdown behavior;
• health reporting and field diagnostics;
• regulatory, material and traceability needs.

Separate must pass requirements from characterization data. A camera that must sustain a defined video stream needs a minimum write floor and maximum stall duration, not only an average MB/s result. An embedded controller may care more about boot latency and power-cycle recovery than peak throughput.

Bind approval to an exact configuration

Record every device under test:

If a later shipment cannot be tied to that identity, the qualification cannot prove equivalence. Pair the release with a PCN/EOL and change-control process and a fixed-BOM requirement where variation is unacceptable.

Build a layered test matrix

1. Interface and compatibility

Test cold boot, warm reset, sleep/resume, removal where applicable, all required host ports and expected protocol modes. Record negotiated link width or speed, capacity and identifiers. Exercise error paths such as timeout, retry and device reset without injecting destructive faults into uncontrolled equipment.

2. Functional and data-integrity behavior

Cover full-address-range write/read verification where feasible, filesystem creation, large and small files, boundary sizes, trim/deallocate or erase behavior and application data structures. Verify that the system detects a failed comparison rather than only recording that the command completed.

3. Workload performance

Use the application’s block sizes, read/write mix, queue depth, access pattern, sync behavior and duty cycle. Report throughput, IOPS and latency percentiles over time. Avoid approving from a single best run.

SNIA’s performance specifications exist to make solid-state measurements accurate, repeatable and comparable [1]. Micron’s measurement guidance follows a flow of restoring a known state, preconditioning, testing into steady state and recording results from fresh through steady behavior [2].

4. Environment and mechanics

Run the required workload at defined ambient and device temperatures in the final enclosure. Include airflow, heatsink, vibration or connector retention where the product requirement calls for them. For NVMe designs, correlate performance with the thermal-throttling counters rather than accepting one temperature screenshot.

5. Power and recovery

Test normal shutdown, repeated power cycle, brownout or unexpected removal using an approved fixture and safe procedure. Verify bootability, filesystem consistency, application recovery and health logs after each sequence. Power-loss testing can damage data by design; isolate samples and preserve results.

6. Endurance and health

Map the estimated field workload to host writes, write amplification and service life. Record a baseline of supported SMART, NVMe or eMMC health fields, then trend them during stress. The WAF guide explains why the same host-write volume can create different media wear.

Preconditioning prevents a false pass

Managed flash often starts fast. Empty blocks and a temporary SLC cache can make a fresh device look better than it behaves after sustained use.

SNIA defines workload-independent and workload-dependent preconditioning to move an SSD toward steady state [1]. Micron describes sequentially writing twice the advertised capacity in its example workload-independent procedure, followed by metric-specific testing into steady state [2].

Do not copy that SSD procedure unchanged to every microSD or eMMC. Preserve the principle:

1. Define the starting state.
2. Fill or exercise the address range in a documented way.
3. Apply the representative workload long enough for behavior to converge.
4. Measure the window that represents service, not only the first minutes.

If the product uses a burst cache, include a transfer larger than the cache and report both burst and sustained write speed.

Measure distributions and transitions

Average latency can hide stalls that break an application. Retain:

• median, p95, p99, p99.9 and maximum latency where the workload supports them;
• performance versus time, temperature and bytes written;
• transition from fresh to steady state;
• recovery after idle, trim/deallocate and workload changes;
• device errors, retries, resets and thermal events.

NVM Express distinguishes protocol compliance from customer qualification: meeting a particular latency may be an end-user requirement even when it is not a compliance item [3]. Compliance certificates are useful inputs, not substitutes for the final host test.

Control samples and failures

Use production-representative samples. Engineering samples, hand-selected units or devices with undocumented firmware do not establish production capability.

For a failure:

1. Stop destructive retesting.
2. Preserve the exact device, host logs, workload and time of failure.
3. Record device identity and health/error logs using read-only methods where possible.
4. Reproduce on a controlled sample only when the plan allows it.
5. Request supplier failure analysis with a defined question and evidence package.

NVMe-CLI can read identity, SMART/Health, error and self-test information where supported [4]. Store raw outputs, not only screenshots or a pass/fail summary.

Define the release package

A production approval should contain:

• approved part number and configuration;
• qualification plan and revision;
• sample and lot table;
• calibrated equipment and software versions;
• raw data, plots and test logs;
• pass/fail limits and results;
• deviations, risk acceptance and owners;
• approved golden sample if one is used;
• effective production lot/date;
• requalification triggers.

Triggers commonly include controller, NAND or firmware changes, manufacturing transfer, capacity change, major host revision, repeated field failure or an unapproved substitution.

Bottom line

Flash qualification is a controlled release decision, not a speed screenshot. Bind approval to the exact device and host, test the real workload after preconditioning, measure tail behavior across environment and power events, and preserve auditable evidence. Then connect production shipments to the qualified baseline through traceability and change control.