Pulumi Programming Model
Internal
Overview
In Pulumi, infrastructure resources are described programmatically in a program, that gets executed when the pulumi preview and pulumi up are run. At the end of the program execution, the desired state of the infrastructure is built in memory and it is compared with the actual state, pulled from the Pulumi backend, specifically from the state file associated with the active stack the program is being executed in the context of.
Before any interaction with the infrastructure platform could takes place, Pulumi needs to build a context to maintain a model of the infrastructure resources, as they exist in the infrastructure platform backend - the actual state. That is the stack. A stack contains metadata and a state file, which has the actual state of the resources in the stack, as known at the moment. This is how an empty stack is created. Note that the state file maintained by a stack is a cache of the actual state of the infrastructure platform backend, and it can be synchronized with pulumi refresh. Depending on delta between the desired state and the actual state, the Pulumi client takes various actions: infrastructure resources are created, updated or deleted. It is the Pulumi client that interacts with the infrastructure platform, so proper access credentials must exist in the environment that runs the Pulumi client.
While the rest of this article uses Python as example, Pulumi offers SDKs for Python, TypeScript and Go.
Creating an Empty Stack
A project may contain several stacks out of which one is active.
To list all stacks in the project:
pulumi stack ls
More details on listing stacks:
To create an empty stack in the project:
pulumi stack init <org-name>/<stack-name>
More details on creating empty stacks:
The stack metadata in a raw format can be displayed with:
pulumi stack export
More details about exporting stacks:
Creating an Infrastructure Resource in Code
Starting with an empty stack, the simplest way to provision an infrastructure resource is to write the code that creates an instance of that resource, using the API exposed by the dedicated resource provider.
This is an example of a Python program, saved in a __main__.py in a Pulumi project:
import pulumi_datadog
dashboard_json = """
{
"title": "Blue",
"layout_type": "ordered",
"reflow_type": "fixed",
"widgets": [
{
"definition": {
"color": "#4d4d4d",
"font_size": "auto",
"text": "This is some sample text.",
"text_align": "left",
"type": "free_text"
},
"layout": {
"height": 1,
"width": 2,
"x": 0,
"y": 0
}
}
]
}
"""
pulumi_datadog.DashboardJson("Dashboard-01", dashboard=dashboard_json)
Upon the execution of the pulumi up command, the Pulumi runtime builds the desired state in memory and asks for the permission to perform the update:
pulumi up
Previewing update (aiml-dp/ovidiu-experimental-datadog-dashboard-from-json/main)
Type Name Plan
+ pulumi:pulumi:Stack ovidiu-experimental-datadog-dashboard-from-json-main create
+ └─ datadog:index:DashboardJson Dashboard-01 create
Resources:
+ 2 to create
Do you want to perform this update? [Use arrows to move, enter to select, type to filter]
yes
> no
details
If the permission is granted, Pulumi reconciles the desired state (one resource) and actual state (no resource) and crates the resource in the infrastructure platform backend. At the same time, the stack state is updated.
Updating an Infrastructure Resource in Code
Upon the initial infrastructure resource creation, it state exists in three places: in the infrastructure platform itself, in the stack state file and in the program. To update the infrastructure resource from the program, update the code, and run pulumi up again. We change the JSON representation and update the color to dark blue:
import pulumi_datadog
dashboard_json = """
{
"title": "Blue",
...
"widgets": [
{
"definition": {
"color": "#00008B",
...
},
...
]
}
"""
pulumi_datadog.DashboardJson("Dashboard-01", dashboard=dashboard_json)
pulumi up will build the desired state (which includes the dark blue color) in memory, will compare it with the stored actual state, and will detect differences:
pulumi up
Previewing update (aiml-dp/ovidiu-experimental-datadog-dashboard-from-json/main)
Type Name Plan Info
pulumi:pulumi:Stack ovidiu-experimental-datadog-dashboard-from-json-main
~ └─ datadog:index:DashboardJson Dashboard-01 update [diff: ~dashboard]
Resources:
~ 1 to update
1 unchanged
Do you want to perform this update? [Use arrows to move, enter to select, type to filter]
yes
> no
details
If allowed, pulumi up will initiate infrastructure resource update. At the end of the process, all three states stored in code, stack and infrastructure platform will be reconciled.
Importing (Adopting) an Infrastructure Resource
An existing infrastructure resource can be imported (adopted) into an existing stack. This action allows it to be managed by Pulumi via the usual Pulumi workflows.
Assuming an empty stack and a "Green" dashboard that displays some green text in a widget, the import process consists of running the pulumi import and reconciling the state of the program with the infrastructure resource state.
To run pulumi import, the infrastructure platform resource ID is required.
pulumi import datadog:index/dashboardJson:DashboardJson 'adopted-dashboard-01' aeu-qi8-d2c
After import in the stack state, it is very important to also update the program with the same state. The exact code to use is provided by the pulumi import command at stdout:
...
Resources:
+ 1 created
= 1 imported
2 changes
Duration: 2s
Please copy the following code into your Pulumi application. Not doing so
will cause Pulumi to report that an update will happen on the next update command.
Please note that the imported resources are marked as protected. To destroy them
you will need to remove the `protect` option and run `pulumi update` *before*
the destroy will take effect.
import pulumi
import pulumi_datadog as datadog
adopted_dashboard_01 = datadog.DashboardJson("adopted-dashboard-01",
dashboard="{\"description\":null,\"layout_type\":\"ordered\",\"notify_list\":[],\"reflow_type\":\"fixed\",\"restricted_roles\":[],\"template_variables\":[],\"title\":\"Green\",\"widgets\":[{\"definition\":{\"color\":\"#41c464\",\"font_size\":\"auto\",\"text\":\"This is some green text.\",\"text_align\":\"left\",\"type\":\"free_text\"},\"layout\":{\"height\":1,\"width\":2,\"x\":0,\"y\":0}}]}",
url="/dashboard/aeu-qi8-d2c/green",
opts=pulumi.ResourceOptions(protect=True))
In consequence, the program should be updated as follows:
import pulumi
import pulumi_datadog as datadog
adopted_dashboard_01 = datadog.DashboardJson(
"adopted-dashboard-01",
dashboard="{\"description\":null,\"layout_type\":\"ordered\",\"notify_list\":[],"
"\"reflow_type\":\"fixed\",\"restricted_roles\":[],\"template_variables\":[],"
"\"title\":\"Green\",\"widgets\":[{\"definition\":{\"color\":\"#41c464\","
"\"font_size\":\"auto\",\"text\":\"This is some green text.\","
"\"text_align\":\"left\",\"type\":\"free_text\"},\"layout\":{\"height\":1,"
"\"width\":2,\"x\":0,\"y\":0}}]}",
url="/dashboard/aeu-qi8-d2c/green",
opts=pulumi.ResourceOptions(protect=True))
If the program is not updated as such, when pulumi up is executed the next time, the in-memory model will not contain the definition of "adopted-dashboard-01", which exists in the stack state and the infrastructure platform state, so Pulumi will attempt to delete it from the infrastructure platform.
Note that using:
id = 'aeu-qi8-d2c'
datadog.DashboardJson.get('adopted-dashboard-01', id)
is not a good idea. This function all will update the stack state and make 'adopted-dashboard-01' "external", which is not what we want.
For more details on importing individual resources into a stack see:
Deleting an Infrastructure Resource in Code
To delete an infrastructure resource from code, simply remove from the program the API calls that instantiates that resource. Upon execution, that instance will be missing from the desired state, and Pulumi will remove it from the infrastructure platform and synchronize the stack state.