Salesforce DX

This post provides a practioner’s introduction to Salesforce DX (Developer Experience) a new software development lifecycle paradigm for Salesforce. Please note at the time of writing (Spring ’17 – v40.0) Salesforce DX is at public beta status.

Salesforce DX (or SFDX) – What is it?

A new set of tools and practices to accelerate the software development lifecycle on the Salesforce platform. The tools in question enable established (technology agnostic) software development models to be applied to the Salesforce context efficiently; the practices provide guidance in how to do this effectively and at scale.
Many aspects of Salesforce DX have become standard practice over the last few years; source-driven development using tools such as Jenkins and git being a good example. Other aspects such as artifact-based development, the CLI and Scratch Orgs introduce a fundamentally new paradigm.

The Salesforce development model has historically been org-centric in concept; with the entire set of metadata types representing a single monolithic code-base. This conceptual model has benefits in respect to ensuring complex interdependencies are not overlooked and code structure and conventions are considered fully in a standardised approach. In the org-centric model – and typically not exclusively – a source-code control repository is mapped at the org-level and deployments are org-based not feature or app based. Salesforce DX provides the tools and practices to change the development model from org-centric to artifact-based; where an artifact represents a collection of components (org subset) – developed, tested and deployed together in safe isolation.

SFDX Tools

CLI (Command-line Interface – sfdx)

The new SFDX CLI provides the foundation for Salesforce DX with capabilities that span authorisation, org management, metadata creation, data population, packaging and deployment. The CLI commands are extensive and clearly structured within the top-level force namespace.

command examples:

//CLI create a new project (specific DX folder structure plus config file)
sfdx force:project:create

//CLI needs to authorise against the DevHub (OAuth web-based flow or for CI JWT Bearer flow)
sfdx force:auth:web:login -d -a "Hub Org"

//CLI create scratch org (or set existing as default)
sfdx force:org:create -s -f config/project-scratch-def.json

//CLI push source code from local (respects the .forceignore file placed in root folder)
//Conflicts can be overwritten --forceoverwrite
//Note - Local project is the source of truth.
sfdx force:source:push

//CLI run unit tests
sfdx force:apex:test:run

A key concept is the local project which is the primary source of truth. The project structure is different from the Metadata API folder structure with higher-level metadata types such as Object broken down into sub-folders and individual files for sub-types such as listViews and fields. This new sfdx structure (or source shape) is designed for development and source-code control system interactions whereas the longstanding Metadata API structure related to deployment primarily. Conversion between the 2 formats is necessary when creating a new sfdx project from an existing org (or simply from existing metadata) and prior to deploying a sfdx project to a Salesforce org (not Scratch Orgs). The CLI provides commands to perform the format changes as illustrated below.

//Create DX project > Create folder for metadata files then ..

// Retrieve unpackaged metadata
sfdx force:mdapi:retrieve -r ./mdapipkg -u <username> -k ./package.xml

// Retrieve the metadata for a package
sfdx force:mdapi:retrieve -s -r ./mdapipkg -u <username> -p <package name>

// Convert to SFDX format.
sfdx force:mdapi:convert --rootdir <retrieve folder name>

// Convert to Metadata API format.
sfdx force:source:convert -d <metadata folder name>/ --packagename package_name

Scratch Orgs

A Scratch Org is a temporary org that can be created quickly and configured with predefined settings and preferences (Edition, languages, platform features etc.) via a JSON definition file. Scratch Orgs are easily created and deleted via the CLI; up to 50 can be created per-day (per Dev Hub) with up to 25 active concurrently, deletion occurs automatically after 7 days.

A Scratch Org is typically used for development and testing purposes and plays an enabling role in build automation. Note, unlike sandboxes, Scratch Orgs are not constrained by a refresh cycle, but also are not a clone of a source org configuration. A typical development workflow will be to create a short-lived feature branch in the Source Code Control system (SCC), create a local sfdx project (from the branch metadata) and Scratch Org, complete development and testing in the Scratch Org, commit change back to SCC and convert and deploy the change to Sandbox. In a Continuous Integration (CI) environment the CI server (Travis CI, Jenkins etc.) may initiate an automated build from the commit event. In such a scenario, headless Scratch Orgs may be used for automated testing (across editions, languages etc.).

The SFDX CLI provides commands for automating the population of data in a Scratch Org. This can be via CSV file loading or more interestingly via org-to-org copy via SOQL query export and subsequent import.

Finally, a key feature of the CLI is the automated tracking of change (the diff) between the current local project and the default Scratch Org. The CLI push and pull commands that facilitate the movement of change between the local project and the Scratch Org automatically process changed components only.

Dev Hub

The SFDX CLI must authorise to a Dev Hub before Scratch Orgs can be created; each Scratch Org is related to a parent Dev Hub. The Dev Hub is a production Salesforce org (Enterprise Edition or Unlimited Edition) – the feature can be activated from the Setup menu. A Dev Hub trial org can be created to enable safe experimentation with SFDX. Dev Hub trial orgs expire after 30 days and are limited (40 Scratch Orgs created per day, 20 active concurrently) why this is subtly different from the standard limits is unclear.

Authorisation of the Dev Hub org (and indeed any org) by the SFDX CLI is possible via OAuth Web Flow or JWT-based Flow. The latter being preferred for system authentications such as those conducted by a CI server.

For ISV (and others) working with namespaces, the Dev Hub provides a Namespace Registry capability which (as the name suggests) allows Developer Editions containing namespaces to be registered. Once the org link is complete, Scratch Orgs can be created as normal. This capability is great for ISV developing in a namespace registered org (as opposed to utilising a separate packaging org to apply the namespace).

IDE2

The new Force.com IDE2 is an Eclipse plug-in built on the SFDX CLI. Additional features of the IDE2 include the Lightning Code Analyser and Apex Code Analysis tools. The former wraps the capabilities of ESLint and the Salesforce Lightning CLI to evaluate JavaScript code against standard or custom rule sets. The latter covers syntax error detection and highlighting plus code auto-completion. The IDE2 also provides an embedded browser for editing Lightning Pages.

On initial inspection there didn’t appear to be any means to directly login/import metadata from an existing org. Presumably this must be done externally via the SFDX CLI.

Finally, the new IDE2 provides integrated tools for working with Git repositories, a key facet of the envisaged developer workflows centred around Salesforce DX.

Summary

At the time of writing, the available SFDX documentation relates primarily to the technical aspects of the tools, there is minimal information relating to process guidance (or practices). Whilst it’s relatively easy to envisage how the new tools and techniques could be integrated into development, test and deployment workflows, best-practice guidance in such areas would certainly provide confidence and accelerate adoption.

Historically, a key sticking point for source-based development (and deployment in general) in the Salesforce context has been the Metadata API coverage. In short, not all metadata types are supported and therefore automated deployments (whether Change Set or API based) are typically accompanied by pre- and post- deployment steps that manually address the gaps. As there’s no significant change in this regard, it’s likely that Scratch Orgs will require the same approach – however considering Scratch Org use cases will typically be code related this may have limited actual impact.

In general however Salesforce DX is a considerable step forward in terms of applying established software development lifecycle practices to the Salesforce context. The CLI alone is incredibly impressive and provides significant flexibility in respect to development, testing and release process. It will be interesting to see how tools vendors react to the availability of the CLI as a foundation for future, evolved IDE capabilities. Hopefully Salesforce DX represents the first step of many in this direction.