Examples by Use Case#

本文将详细介绍如何使用 aws_ssm_run_command 库来实现各种常见的 Use Cases.

Prepare#

First, let’s create a boto session and then launch an EC2 instance for testing. Make sure this EC2 meet the pre-requisite mentioned in the How it Work section.

Then we are going to define some helper functions for this example.

[3]:

import uuid
import json
from pathlib import Path

from rich import print as rprint
from boto_session_manager import BotoSesManager
from s3pathlib import S3Path, context

import aws_ssm_run_command.api as aws_ssm_run_command

# set boto session
bsm = BotoSesManager(profile_name="bmt_app_dev_us_east_1")
# set two instance ids
inst_id_1 = "i-00d17e6620f53ea14"
inst_id_2 = "i-04263cc722e9b0ac3"
# set an S3 bucket location for storing the output
s3dir = S3Path(f"s3://{bsm.aws_account_alias}-{bsm.aws_region}-data/projects/aws_ssm_run_command/example/")
# set the current working directory
dir_here = Path.cwd().absolute()
# tell s3pathlib to use the given boto session
context.attach_boto_session(bsm.boto_ses)

[4]:

def rprint_response(res):
    """
    A helper function to print the response of boto3 API.
    """
    if "ResponseMetadata" in res:
        del res["ResponseMetadata"]
    rprint(res)

def s3url_to_s3uri(url: str) -> str:
    """
    Convert https://s3.amazonaws.com/bucket/key to s3://bucket/key
    """
    return "s3://" + url.split("/", 3)[-1]

Use Case 1. Send one command to one EC2 instance#

这是一个最简单的例子, 我们只将一条简单的 echo "random-uuid" > ~/uuid.txt 命令发送到一台 EC2 上. 运行之后, 我们再用 cat ~/uuid.txt 命令打印这个文件的内容. 这里最关键的函数是 run_shell_script_sync(), 它是对 boto3.client(“ssm”).send_command() 和 boto3.client(“ssm”).get_command_invocation() 的封装. 它会发送一个命令, 然后等待命令执行完毕. 它返回命令执行的详细信息. 该函数会返回一个 CommandInvocation 对象, 你可以从这个对象中获取 return code, stdout, stderr 等信息.

如果你想将发送命令和等待分开进行, 你也可以用 run_shell_script_async() 和 wait_until_send_command_succeeded() 来实现, 不过这里我们就不展示了.

[37]:

value = uuid.uuid4()
cmd = f"echo {value} > ~/uuid.txt"
print("--- uuid value ---")
rprint(value)

print("--- command ---")
rprint(cmd)

--- uuid value ---

4441c17d-08a0-4156-b6ef-13364fbeb4e6

--- command ---

echo 4441c17d-08a0-4156-b6ef-13364fbeb4e6 > ~/uuid.txt

[6]:

print("--- send command and wait it to succeed ---")
command_invocations = aws_ssm_run_command.better_boto.run_shell_script_sync(
    ssm_client=bsm.ssm_client,
    commands=cmd,
    instance_ids=inst_id_1,
)

--- send command and wait it to succeed ---
start waiter, polling every 3 seconds, timeout in 60 seconds.
on 0 th attempt, elapsed 0 seconds, remain 60 seconds ...

[7]:

print("--- command invocation details ---")
rprint(command_invocations)

--- command invocation details ---

[
    CommandInvocation(
        CommandId='599ed08c-45c3-4267-ade3-df3712dcc024',
        InstanceId='i-00d17e6620f53ea14',
        Comment='',
        DocumentName='AWS-RunShellScript',
        DocumentVersion='1',
        PluginName='aws:runShellScript',
        ResponseCode=0,
        ExecutionStartDateTime='2024-06-18T03:53:38.143Z',
        ExecutionElapsedTime='PT0.006S',
        ExecutionEndDateTime='2024-06-18T03:53:38.143Z',
        Status='Success',
        StatusDetails='Success',
        StandardOutputContent='',
        StandardOutputUrl='',
        StandardErrorContent='',
        StandardErrorUrl='',
        CloudWatchOutputConfig={'CloudWatchLogGroupName': '', 'CloudWatchOutputEnabled': False}
    )
]

[8]:

print("--- send another command and wait it to succeed ---")
cmd = "cat ~/uuid.txt"
rprint(cmd)
command_invocations = aws_ssm_run_command.better_boto.run_shell_script_sync(
    ssm_client=bsm.ssm_client,
    commands=cmd,
    instance_ids=inst_id_1,
)

--- send another command and wait it to succeed ---

cat ~/uuid.txt

start waiter, polling every 3 seconds, timeout in 60 seconds.
on 0 th attempt, elapsed 0 seconds, remain 60 seconds ...

[9]:

print("--- stdout ---")
stdout = command_invocations[0].StandardOutputContent.strip()
rprint(f"stdout = {stdout}, uuid = {value}")

--- stdout ---

stdout = f2266cf8-96ed-46b3-a07f-2eff5c6a1d85, uuid = f2266cf8-96ed-46b3-a07f-2eff5c6a1d85

Use Case 2. Get Return Code, stdout, stderr#

任何一条 Command 都会返回一个 return code, 通常 0 表示成功, 非 0 表示失败. 如果成功, 命令可能会打印一些信息到 stdout, 例如 python --version 会打印版本号. 如果失败, 命令可能会打印一些信息到 stderr. 例如你运行 python my_script.py 中抛出了异常信息. 对于关键的业务场景, 我们通常不能仅仅把命令发出去就不管了, 而要对 return code, stdout, stderr 进行处理. 下面我们来看看如何获取这些信息.

这里我们先快速介绍一下 Python 中 subprocess 模块中的 CompletedProcess 对象. subprocess 是 Python 中用来运行命令行的模块. 每个 subprocess.run() 都会返回一个 CompletedProcess, 你可以从这个对象中获得 return code, stdout, stderr.

而 boto3.client(“ssm”).get_command_invocation() API 的 response 中有 ResponseCode, StandardOutputContent, StandardErrorContent 字段, 分别对应 return code, stdout, stderr.

其实在 Use Case 1. Send one command to one EC2 instance 的例子中我们已经展示了如何获得这些信息. 这里再来看一个非常简单的例子.

[10]:

print("--- command ---")
cmd = "python3 --version"
rprint(cmd)

--- command ---

python3 --version

[11]:

print("--- send command and wait it to succeed ---")
command_invocations = aws_ssm_run_command.better_boto.run_shell_script_sync(
    ssm_client=bsm.ssm_client,
    commands=cmd,
    instance_ids=inst_id_1,
)

--- send command and wait it to succeed ---
start waiter, polling every 3 seconds, timeout in 60 seconds.
on 0 th attempt, elapsed 0 seconds, remain 60 seconds ...

[12]:

print("--- command invocation details ---")
rprint(command_invocations)

--- command invocation details ---

[
    CommandInvocation(
        CommandId='ebf0307f-a271-4118-9a3b-265a8fb4b75d',
        InstanceId='i-00d17e6620f53ea14',
        Comment='',
        DocumentName='AWS-RunShellScript',
        DocumentVersion='1',
        PluginName='aws:runShellScript',
        ResponseCode=0,
        ExecutionStartDateTime='2024-06-18T03:53:40.564Z',
        ExecutionElapsedTime='PT0.006S',
        ExecutionEndDateTime='2024-06-18T03:53:40.564Z',
        Status='Success',
        StatusDetails='Success',
        StandardOutputContent='Python 3.9.16\n',
        StandardOutputUrl='',
        StandardErrorContent='',
        StandardErrorUrl='',
        CloudWatchOutputConfig={'CloudWatchLogGroupName': '', 'CloudWatchOutputEnabled': False}
    )
]

[13]:

print("--- stdout ---")
rprint(command_invocations[0].StandardOutputContent.strip())

--- stdout ---

Python 3.9.16

Use Case 3. Send Large stdout or stderr to AWS S3#

有的时候一个命令的输出内容可能会非常大, 非常复杂, 甚至超过了 SSM 的 24,000 characters 的限制. 做出这个限制是因为 AWS SSM API 本质上是一个 HTTP request, 一般 API 服务器为了避免负载过高, 都会对 response 中的 payload 大小做出限制. 为了解决这个问题, AWS 的 boto3.client(“ssm”).send_command() API 中有 OutputS3BucketName 和 OutputS3KeyPrefix 参数, 可以让你将 stdout 和 stderr 输出到 S3 中. 这样你就可以在 S3 中下载这些内容, 而不是直接从 API response 中获取.

[14]:

# clean up the s3 folder to ensure a fresh start
s3dir.delete()

[14]:

S3Path('s3://bmt-app-dev-us-east-1-data/projects/aws_ssm_run_command/example/')

[15]:

print("--- command ---")
cmd = "python3 --version"
rprint(cmd)

--- command ---

python3 --version

[16]:

print("--- send command and wait it to succeed ---")
command_invocations = aws_ssm_run_command.better_boto.run_shell_script_sync(
    ssm_client=bsm.ssm_client,
    commands=cmd,
    instance_ids=inst_id_1,
    output_s3_bucket_name=s3dir.bucket,
    output_s3_key_prefix=s3dir.key,
)

--- send command and wait it to succeed ---
start waiter, polling every 3 seconds, timeout in 60 seconds.
on 0 th attempt, elapsed 0 seconds, remain 60 seconds ...

[17]:

print("--- command invocation details ---")
rprint(command_invocations)

--- command invocation details ---

[
    CommandInvocation(
        CommandId='88f5b32a-365e-4f26-a7d2-c65d48dbfac1',
        InstanceId='i-00d17e6620f53ea14',
        Comment='',
        DocumentName='AWS-RunShellScript',
        DocumentVersion='1',
        PluginName='aws:runShellScript',
        ResponseCode=0,
        ExecutionStartDateTime='2024-06-18T03:53:42.065Z',
        ExecutionElapsedTime='PT0.105S',
        ExecutionEndDateTime='2024-06-18T03:53:42.065Z',
        Status='Success',
        StatusDetails='Success',
        StandardOutputContent='Python 3.9.16\n',
        StandardOutputUrl='https://s3.us-east-1.amazonaws.com/bmt-app-dev-us-east-1-data/projects/aws_ssm_run_comma
nd/example/88f5b32a-365e-4f26-a7d2-c65d48dbfac1/i-00d17e6620f53ea14/awsrunShellScript/0.awsrunShellScript/stdout',
        StandardErrorContent='',
        StandardErrorUrl='https://s3.us-east-1.amazonaws.com/bmt-app-dev-us-east-1-data/projects/aws_ssm_run_comman
d/example/88f5b32a-365e-4f26-a7d2-c65d48dbfac1/i-00d17e6620f53ea14/awsrunShellScript/0.awsrunShellScript/stderr',
        CloudWatchOutputConfig={'CloudWatchLogGroupName': '', 'CloudWatchOutputEnabled': False}
    )
]

[18]:

print("--- s3 dir files ---")
for s3path in s3dir.iter_objects():
    rprint(s3path.uri)

--- s3 dir files ---

s3://bmt-app-dev-us-east-1-data/projects/aws_ssm_run_command/example/88f5b32a-365e-4f26-a7d2-c65d48dbfac1/i-00d17e6
620f53ea14/awsrunShellScript/0.awsrunShellScript/stdout

[19]:

s3path = S3Path(s3url_to_s3uri(command_invocations[0].StandardOutputUrl))
rprint(f"read stdout from {s3path.uri}:")
print("--- stdout ---")
rprint(s3path.read_text().strip())

read stdout from
s3://bmt-app-dev-us-east-1-data/projects/aws_ssm_run_command/example/88f5b32a-365e-4f26-a7d2-c65d48dbfac1/i-00d17e6
620f53ea14/awsrunShellScript/0.awsrunShellScript/stdout:

--- stdout ---

Python 3.9.16

Use Case 4. Send Multiple Commands to Multiple Instances#

在运维领域, 批量对多台机器按照一定顺序执行多条命令是非常常见的需求. 在以前工程师常用 Ansible (2012 年第一次发布) 来实现这个需求. 而在云时代, AWS SSM 原生提供了这一功能.

这里最关键的函数依然是 run_shell_script_sync(). 如果不用这个函数, boto3.client(“ssm”).send_command() 会返回一个 CommandId 和 InstanceIds (是一个 EC2 instance id 的列表). 然后你要用一个 for 循环便利所有的 instance_id, 然后调用 boto3.client(“ssm”).get_command_invocation() 来轮询这个 CommandId 直到命令执行完毕. 最终才能获得每个 EC2 instance 的 command invocation result. 这个代码非常繁琐. 而 run_shell_script_sync 已经帮你封装好了.

[20]:

# clean up the s3 folder to ensure a fresh start
s3dir.delete()

[20]:

S3Path('s3://bmt-app-dev-us-east-1-data/projects/aws_ssm_run_command/example/')

[21]:

print("--- command ---")
commands = [
    "aws --version",
    "python3 --version",
]
rprint(commands)

--- command ---

['aws --version', 'python3 --version']

[22]:

print("--- send command and wait it to succeed ---")
command_invocations = aws_ssm_run_command.better_boto.run_shell_script_sync(
    ssm_client=bsm.ssm_client,
    commands=commands,
    instance_ids=[inst_id_1, inst_id_2],
    output_s3_bucket_name=s3dir.bucket,
    output_s3_key_prefix=s3dir.key,
)

--- send command and wait it to succeed ---
start waiter, polling every 3 seconds, timeout in 60 seconds.
on 1 th attempt, elapsed 3 seconds, remain 57 seconds ...
start waiter, polling every 3 seconds, timeout in 60 seconds.
on 0 th attempt, elapsed 0 seconds, remain 60 seconds ...

[23]:

print("--- command invocation details ---")
rprint(command_invocations)

--- command invocation details ---

[
    CommandInvocation(
        CommandId='a4621bd0-192f-41f5-a891-88a5d91eeeac',
        InstanceId='i-00d17e6620f53ea14',
        Comment='',
        DocumentName='AWS-RunShellScript',
        DocumentVersion='1',
        PluginName='aws:runShellScript',
        ResponseCode=0,
        ExecutionStartDateTime='2024-06-18T03:53:43.547Z',
        ExecutionElapsedTime='PT0.62S',
        ExecutionEndDateTime='2024-06-18T03:53:43.547Z',
        Status='Success',
        StatusDetails='Success',
        StandardOutputContent='aws-cli/2.15.30 Python/3.9.16 Linux/6.1.92-99.174.amzn2023.x86_64 
source/x86_64.amzn.2023 prompt/off\nPython 3.9.16\n',
        StandardOutputUrl='https://s3.us-east-1.amazonaws.com/bmt-app-dev-us-east-1-data/projects/aws_ssm_run_comma
nd/example/a4621bd0-192f-41f5-a891-88a5d91eeeac/i-00d17e6620f53ea14/awsrunShellScript/0.awsrunShellScript/stdout',
        StandardErrorContent='',
        StandardErrorUrl='https://s3.us-east-1.amazonaws.com/bmt-app-dev-us-east-1-data/projects/aws_ssm_run_comman
d/example/a4621bd0-192f-41f5-a891-88a5d91eeeac/i-00d17e6620f53ea14/awsrunShellScript/0.awsrunShellScript/stderr',
        CloudWatchOutputConfig={'CloudWatchLogGroupName': '', 'CloudWatchOutputEnabled': False}
    ),
    CommandInvocation(
        CommandId='a4621bd0-192f-41f5-a891-88a5d91eeeac',
        InstanceId='i-04263cc722e9b0ac3',
        Comment='',
        DocumentName='AWS-RunShellScript',
        DocumentVersion='1',
        PluginName='aws:runShellScript',
        ResponseCode=0,
        ExecutionStartDateTime='2024-06-18T03:53:43.563Z',
        ExecutionElapsedTime='PT0.623S',
        ExecutionEndDateTime='2024-06-18T03:53:43.563Z',
        Status='Success',
        StatusDetails='Success',
        StandardOutputContent='aws-cli/2.15.30 Python/3.9.16 Linux/6.1.92-99.174.amzn2023.x86_64 
source/x86_64.amzn.2023 prompt/off\nPython 3.9.16\n',
        StandardOutputUrl='https://s3.us-east-1.amazonaws.com/bmt-app-dev-us-east-1-data/projects/aws_ssm_run_comma
nd/example/a4621bd0-192f-41f5-a891-88a5d91eeeac/i-04263cc722e9b0ac3/awsrunShellScript/0.awsrunShellScript/stdout',
        StandardErrorContent='',
        StandardErrorUrl='https://s3.us-east-1.amazonaws.com/bmt-app-dev-us-east-1-data/projects/aws_ssm_run_comman
d/example/a4621bd0-192f-41f5-a891-88a5d91eeeac/i-04263cc722e9b0ac3/awsrunShellScript/0.awsrunShellScript/stderr',
        CloudWatchOutputConfig={'CloudWatchLogGroupName': '', 'CloudWatchOutputEnabled': False}
    )
]

[24]:

print("--- s3 dir files ---")
for s3path in s3dir.iter_objects():
    rprint(s3path.uri)

--- s3 dir files ---

s3://bmt-app-dev-us-east-1-data/projects/aws_ssm_run_command/example/a4621bd0-192f-41f5-a891-88a5d91eeeac/i-00d17e6
620f53ea14/awsrunShellScript/0.awsrunShellScript/stdout

s3://bmt-app-dev-us-east-1-data/projects/aws_ssm_run_command/example/a4621bd0-192f-41f5-a891-88a5d91eeeac/i-04263cc
722e9b0ac3/awsrunShellScript/0.awsrunShellScript/stdout

[25]:

for command_invocation in command_invocations:
    s3path = S3Path(s3url_to_s3uri(command_invocation.StandardOutputUrl))
    rprint(f"read stdout from {s3path.uri}:")
    print("--- stdout ---")
    rprint(s3path.read_text().strip())

read stdout from
s3://bmt-app-dev-us-east-1-data/projects/aws_ssm_run_command/example/a4621bd0-192f-41f5-a891-88a5d91eeeac/i-00d17e6
620f53ea14/awsrunShellScript/0.awsrunShellScript/stdout:

--- stdout ---

aws-cli/2.15.30 Python/3.9.16 Linux/6.1.92-99.174.amzn2023.x86_64 source/x86_64.amzn.2023 prompt/off
Python 3.9.16

read stdout from
s3://bmt-app-dev-us-east-1-data/projects/aws_ssm_run_command/example/a4621bd0-192f-41f5-a891-88a5d91eeeac/i-04263cc
722e9b0ac3/awsrunShellScript/0.awsrunShellScript/stdout:

--- stdout ---

aws-cli/2.15.30 Python/3.9.16 Linux/6.1.92-99.174.amzn2023.x86_64 source/x86_64.amzn.2023 prompt/off
Python 3.9.16

Use Case 5. Error Handling For Multiple Commands on Multiple Instances#

在实际生产环境中, 不可能事事一帆风顺. 而对于异常处理, 我们要考虑如下维度:

在同一台机器上执行多条命令时, 是顺序执行还是并行执行?
在多台机器上执行命令时, 是一台机器执行完了再执行下一台 (顺序执行), 还是所有机器同时执行?
当一条命令失败时, 是否要继续执行后续的命令?
当一台机器执行失败时, 是否要继续执行后续的机器?

当这些维度组合在一起时, 异常处理就不那么容易了.

run_shell_script_sync() 函数提供了 allow_fails_config 参数, 可以让你精确控制异常处理. 这个参数的值可以是:

如果是 0, 任何一条命令失败, 都会抛出异常. 这也是默认设置.
如果是 1 或者大于 1 的整数, 则允许在 N 台 instance 上失败.
如果是 0 到 1 之间的浮点数 (不能是 0.0 或者 1.0), 则允许一定比例的 instance 失败. 结果向下取整.
如果是 str 或是 str 的列表, 那么允许指定的 instance id 失败.

在允许失败且不抛出异常的情况下, 你会看到返回的 CommandInvocation 列表中有的状态是成功, 有的状态是失败.

[26]:

# clean up the s3 folder to ensure a fresh start
s3dir.delete()

[26]:

S3Path('s3://bmt-app-dev-us-east-1-data/projects/aws_ssm_run_command/example/')

[27]:

print("--- command ---")
commands = [
    "python3 --version", # This always work
    # I have run "sudo python3 -m ensurepip --upgrade" in instance 1 already
    "pip3 --version", # This only works on instance 1, instance 2 doesn't have "pip"
]
rprint(commands)

--- command ---

['python3 --version', 'pip3 --version']

[28]:

print("--- send command and wait it to succeed ---")
command_invocations = aws_ssm_run_command.better_boto.run_shell_script_sync(
    ssm_client=bsm.ssm_client,
    commands=commands,
    instance_ids=[inst_id_1, inst_id_2],
    output_s3_bucket_name=s3dir.bucket,
    output_s3_key_prefix=s3dir.key,
    allow_fails_config=2,
)

--- send command and wait it to succeed ---
start waiter, polling every 3 seconds, timeout in 60 seconds.
on 0 th attempt, elapsed 0 seconds, remain 60 seconds ...
start waiter, polling every 3 seconds, timeout in 60 seconds.
on 0 th attempt, elapsed 0 seconds, remain 60 seconds ...

[29]:

print("--- command invocation details ---")
rprint(command_invocations)

--- command invocation details ---

[
    CommandInvocation(
        CommandId='e6101c42-ba78-48f8-ac2f-56eb90913579',
        InstanceId='i-00d17e6620f53ea14',
        Comment='',
        DocumentName='AWS-RunShellScript',
        DocumentVersion='1',
        PluginName='aws:runShellScript',
        ResponseCode=0,
        ExecutionStartDateTime='2024-06-18T03:53:48.217Z',
        ExecutionElapsedTime='PT0.385S',
        ExecutionEndDateTime='2024-06-18T03:53:48.217Z',
        Status='Success',
        StatusDetails='Success',
        StandardOutputContent='Python 3.9.16\npip 21.3.1 from /usr/local/lib/python3.9/site-packages/pip (python 
3.9)\n',
        StandardOutputUrl='https://s3.us-east-1.amazonaws.com/bmt-app-dev-us-east-1-data/projects/aws_ssm_run_comma
nd/example/e6101c42-ba78-48f8-ac2f-56eb90913579/i-00d17e6620f53ea14/awsrunShellScript/0.awsrunShellScript/stdout',
        StandardErrorContent='',
        StandardErrorUrl='https://s3.us-east-1.amazonaws.com/bmt-app-dev-us-east-1-data/projects/aws_ssm_run_comman
d/example/e6101c42-ba78-48f8-ac2f-56eb90913579/i-00d17e6620f53ea14/awsrunShellScript/0.awsrunShellScript/stderr',
        CloudWatchOutputConfig={'CloudWatchLogGroupName': '', 'CloudWatchOutputEnabled': False}
    ),
    CommandInvocation(
        CommandId='e6101c42-ba78-48f8-ac2f-56eb90913579',
        InstanceId='i-04263cc722e9b0ac3',
        Comment='',
        DocumentName='AWS-RunShellScript',
        DocumentVersion='1',
        PluginName='aws:runShellScript',
        ResponseCode=127,
        ExecutionStartDateTime='2024-06-18T03:53:48.235Z',
        ExecutionElapsedTime='PT0.122S',
        ExecutionEndDateTime='2024-06-18T03:53:48.235Z',
        Status='Failed',
        StatusDetails='Failed',
        StandardOutputContent='Python 3.9.16\n',
        StandardOutputUrl='https://s3.us-east-1.amazonaws.com/bmt-app-dev-us-east-1-data/projects/aws_ssm_run_comma
nd/example/e6101c42-ba78-48f8-ac2f-56eb90913579/i-04263cc722e9b0ac3/awsrunShellScript/0.awsrunShellScript/stdout',
        StandardErrorContent='/var/lib/amazon/ssm/i-04263cc722e9b0ac3/document/orchestration/e6101c42-ba78-48f8-ac2
f-56eb90913579/awsrunShellScript/0.awsrunShellScript/_script.sh: line 2: pip3: command not found\nfailed to run 
commands: exit status 127',
        StandardErrorUrl='https://s3.us-east-1.amazonaws.com/bmt-app-dev-us-east-1-data/projects/aws_ssm_run_comman
d/example/e6101c42-ba78-48f8-ac2f-56eb90913579/i-04263cc722e9b0ac3/awsrunShellScript/0.awsrunShellScript/stderr',
        CloudWatchOutputConfig={'CloudWatchLogGroupName': '', 'CloudWatchOutputEnabled': False}
    )
]

[30]:

print("--- s3 dir files ---")
for s3path in s3dir.iter_objects():
    rprint(s3path.uri)

--- s3 dir files ---

s3://bmt-app-dev-us-east-1-data/projects/aws_ssm_run_command/example/e6101c42-ba78-48f8-ac2f-56eb90913579/i-00d17e6
620f53ea14/awsrunShellScript/0.awsrunShellScript/stdout

s3://bmt-app-dev-us-east-1-data/projects/aws_ssm_run_command/example/e6101c42-ba78-48f8-ac2f-56eb90913579/i-04263cc
722e9b0ac3/awsrunShellScript/0.awsrunShellScript/stderr

s3://bmt-app-dev-us-east-1-data/projects/aws_ssm_run_command/example/e6101c42-ba78-48f8-ac2f-56eb90913579/i-04263cc
722e9b0ac3/awsrunShellScript/0.awsrunShellScript/stdout

[31]:

for command_invocation in command_invocations:
    print(f"=== instance id = {command_invocation.InstanceId} ===")
    if command_invocation.is_success():
        s3path = S3Path(s3url_to_s3uri(command_invocation.StandardOutputUrl))
        rprint(f"read stdout from {s3path.uri}:")
        rprint("--- stdout --- ")
        rprint(s3path.read_text().strip())
    else:
        s3path = S3Path(s3url_to_s3uri(command_invocation.StandardOutputUrl))
        rprint(f"read stdout from {s3path.uri}:")
        rprint("--- stdout --- ")
        rprint(s3path.read_text().strip())

        s3path = S3Path(s3url_to_s3uri(command_invocation.StandardErrorUrl))
        rprint(f"read stderr from {s3path.uri}:")
        rprint("--- stderr --- ")
        rprint(s3path.read_text().strip())

=== instance id = i-00d17e6620f53ea14 ===

read stdout from
s3://bmt-app-dev-us-east-1-data/projects/aws_ssm_run_command/example/e6101c42-ba78-48f8-ac2f-56eb90913579/i-00d17e6
620f53ea14/awsrunShellScript/0.awsrunShellScript/stdout:

--- stdout ---

Python 3.9.16
pip 21.3.1 from /usr/local/lib/python3.9/site-packages/pip (python 3.9)

=== instance id = i-04263cc722e9b0ac3 ===

read stdout from
s3://bmt-app-dev-us-east-1-data/projects/aws_ssm_run_command/example/e6101c42-ba78-48f8-ac2f-56eb90913579/i-04263cc
722e9b0ac3/awsrunShellScript/0.awsrunShellScript/stdout:

--- stdout ---

Python 3.9.16

read stderr from
s3://bmt-app-dev-us-east-1-data/projects/aws_ssm_run_command/example/e6101c42-ba78-48f8-ac2f-56eb90913579/i-04263cc
722e9b0ac3/awsrunShellScript/0.awsrunShellScript/stderr:

--- stderr ---

/var/lib/amazon/ssm/i-04263cc722e9b0ac3/document/orchestration/e6101c42-ba78-48f8-ac2f-56eb90913579/awsrunShellScri
pt/0.awsrunShellScript/_script.sh: line 2: pip3: command not found
failed to run commands: exit status 127

Use Case 6. Orchestrate Multiple Commands#

如果你需要执行多条命令, 并且这些命令组成的业务逻辑非常复杂, 那么我推荐用 Python + subprocess 写一个脚本, 在这个脚本中实现复杂的业务逻辑. 而用 SSM Run Command, 将这个脚本从 S3 上下载到 EC2 上执行.

这种方案的好处是, 在 Python 中对 command 的异常处理是非常优雅而容易. 而如果你需要并行执行多个命令, 那么你可以参考这个 subprocess 异步执行的例子. 你获得了最大的灵活性 (你可以在 Python 中做任何事), 同时也能利用 SSM Run Command 的安全性和可靠性的优势.

在下面这个例子中, 我们想要将一个 script.py 脚本上传到 EC2 上执行. 这个脚本会在运行中打印一些信息, 例如一些 log. 然后最后会返回一个字典. 作为测试, 我们也将函数返回的字典打印出来, 并尝试之后恢复这一字典. 这里的一些奇怪的字符是用来测试 AWS SSM 是否能正确地对特殊字符进行编码.

[32]:

path_script = dir_here.joinpath("script.py")
print("--- script source code ---")
rprint(path_script.read_text())

--- script source code ---

# -*- coding: utf-8 -*-

import sys
import json


def run() -> dict:
    print("start")
    print("done")
    return {
        "python": sys.executable,
        "weird_string": "\\a\nb\tc\"d'e@f#g:h/i",
    }


if __name__ == "__main__":
    print(json.dumps(run()))

[33]:

path_aws = "aws"
path_python = "python3"
code = path_script.read_text()
s3uri = s3dir.joinpath("script.py").uri
args = []

print("--- command invocation ---")
command_invocation = aws_ssm_run_command.patterns.run_command_on_one_ec2.run_python_script(
    ssm_client=bsm.ssm_client,
    s3_client=bsm.s3_client,
    instance_id=inst_id_1,
    path_aws=path_aws, # what is the path to the aws cli executable
    path_python=path_python, # what is the path to the Python executable
    code=code, # source code
    s3uri=s3uri, # where you want to upload your code to on S3?
    args=args, # additional argument for ``python your_script.py arg1 arg2 ...``
)
rprint(command_invocation)

--- command invocation ---
start waiter, polling every 3 seconds, timeout in 60 seconds.
on 1 th attempt, elapsed 3 seconds, remain 57 seconds ...

CommandInvocation(
    CommandId='cf85b5bf-c80d-455e-9bdc-2fca51dd7b5d',
    InstanceId='i-00d17e6620f53ea14',
    Comment='',
    DocumentName='AWS-RunShellScript',
    DocumentVersion='1',
    PluginName='aws:runShellScript',
    ResponseCode=0,
    ExecutionStartDateTime='2024-06-18T03:53:49.876Z',
    ExecutionElapsedTime='PT0.849S',
    ExecutionEndDateTime='2024-06-18T03:53:49.876Z',
    Status='Success',
    StatusDetails='Success',
    StandardOutputContent='start\ndone\n{"python": "/usr/bin/python3", "weird_string": 
"\\\\a\\nb\\tc\\"d\'e@f#g:h/i"}\n',
    StandardOutputUrl='',
    StandardErrorContent='',
    StandardErrorUrl='',
    CloudWatchOutputConfig={'CloudWatchLogGroupName': '', 'CloudWatchOutputEnabled': False}
)

[34]:

output_lines = command_invocation.StandardOutputContent.split("\n")
print("--- print function output ---")
rprint(output_lines[:2])

--- print function output ---

['start', 'done']

[35]:

print("--- python function return value ---")
data = json.loads(output_lines[2])
rprint(data)
print("--- python value ---")
rprint(data["python"])
print("--- weird_string value ---")
rprint(data["weird_string"]) # test the string escape

--- python function return value ---

{'python': '/usr/bin/python3', 'weird_string': '\\a\nb\tc"d\'e@f#g:h/i'}

--- python value ---

/usr/bin/python3

--- weird_string value ---

\a
b       c"d'e@f#g:h/i

[ ]: