我想将一些dynamodb表(仅限模式)复制到我的本地环境中以进行测试。首先我尝试过:
aws dynamodb describe-table --table-name Foo > FooTable.json
但很明显,输出模式不符合create-table
命令的输入模式:
aws dynamodb create-table --cli-input-json file://FooTable.json --endpoint=http://localhost:8000
我要避免的是使用aws dynamodb create-table --generate-cli-skeleton
生成数十个骷髅并手动填充它们:/
有没有办法以一种对娱乐“有用”的格式获取表模式?我觉得难以置信的是,通过网络图形界面或标准的aws命令行没有直接的方式 - 在听到他们的服务“好”之后。
答案 0 :(得分:8)
我只是设法完成转储并使用 bchew / dynamodump “恢复”:
git clone git@github.com:bchew/dynamodump.git
请注意文档https://github.com/bchew/dynamodump中的--schemaOnly
选项。命令是:
./dynamodump.py -m backup --schemaOnly --region foo-region --host localhost --srcTable '*' --port 8000 --accessKey fooKey --secretKey barKey
然后,您可以使用-m restore
模式将数据或架构放回本地dynamodb或任何需要的地方:)
话虽如此,我仍然觉得令人难以置信的是亚马逊dynamodb工具链有多糟糕。来吧伙伴们。
答案 1 :(得分:1)
DynamoDB是一个NoSQL数据库,除主键和二级索引外没有架构。您可能会在describe-table中注意到它只显示键和索引,而不显示任何其他列。
我经常编写代码来创建表和键,在某种初始化代码中然后开始使用表。如果桌子已经在那里,它不会受到伤害。稍后如果我尝试插入数据并且“列”不存在则无关紧要,因为DynamoDB会自动创建它。
如果您需要导出数据,there are tools for that将允许您从DynamoDB导出(还有一些允许导入)。
答案 2 :(得分:1)
这需要 aws dynamodb describe-table
输出,并将其转换为 aws dynamodb create-table --cli-input-json
的输入格式:
AWS_PROFILE=xyz aws dynamodb describe-table --table-name MyTable > mytable_full.json
# Pull out just what is minimally needed for table-creation:
#
# TableName
# KeySchema
# AttributeDefinitions (must only contain attributes used in keys)
# Global/Local Secondary Indexes
# Defaults BillingMode to PAY_PER_REQUEST
# (any provisioning can be set up manually based on load)
jq <mytable_full.json '.Table | {TableName, KeySchema, AttributeDefinitions} + (try {LocalSecondaryIndexes: [ .LocalSecondaryIndexes[] | {IndexName, KeySchema, Projection} ]} // {}) + (try {GlobalSecondaryIndexes: [ .GlobalSecondaryIndexes[] | {IndexName, KeySchema, Projection} ]} // {}) + {BillingMode: "PAY_PER_REQUEST"}' >mytable.json
AWS_PROFILE=xyz aws dynamodb create-table --cli-input-json file://mytable.json
您也可以将 json 粘贴到 python 中(python dict 语法与 json 非常匹配)例如
import boto3
dynamodb = boto3.resource("dynamodb")
tabledef = {
"TableName": "MyTable",
"KeySchema": [
...
}
table = dynamodb.create_table(**tabledef)
print("Table status: ", table.table_status)
参考文献:
https://docs.aws.amazon.com/cli/latest/reference/dynamodb/describe-table.html https://docs.aws.amazon.com/cli/latest/reference/dynamodb/create-table.html https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_CreateTable.html https://boto3.amazonaws.com/v1/documentation/api/latest/guide/dynamodb.html#creating-a-new-table
答案 3 :(得分:0)
这是在 Windows 上使用 C#、AWS CLI 和 Newtonsoft JSON 的版本。首先运行以下命令:-
aws dynamodb describe-table --table-name TheTable --profile SourceAWSCredsProfile > TheTable.json
拿起文件,反序列化并序列化为--cli-input-json
友好类:-
TableContainer tableContainer;
string sourceFile = "TheTable.json";
string destFile = "TheTable.cli-input.json";
using (StreamReader file = File.OpenText(sourceFile))
{
JsonSerializer serializer = new JsonSerializer();
tableContainer = (TableContainer)serializer.Deserialize(file, typeof(TableContainer));
}
File.WriteAllText(destFile, JsonConvert.SerializeObject(tableContainer.Table, Formatting.Indented, new JsonSerializerSettings
{
NullValueHandling = NullValueHandling.Ignore
}
));
现在运行这个命令来导入表定义:-
aws dynamodb create-table --cli-input-json file://TheTable.cli-input.json --profile DestinationAWSCredsProfile
TableContainer 类定义如下。某些属性的缺失会清除 --cli-input-json
参数不需要的所有内容。您可以随时通过运行重新创建此类:-
aws dynamodb create-table --generate-cli-skeleton
然后使用 Visual Studio 中非常方便的 Paste Special...
Paste JSON as Classes
功能将输出复制并粘贴到新的类文件中。
public class TableContainer
{
public DynamoTableCLIJSON Table { get; set; }
}
public class DynamoTableCLIJSON
{
public Attributedefinition[] AttributeDefinitions { get; set; }
public string TableName { get; set; }
public Keyschema[] KeySchema { get; set; }
public Localsecondaryindex[] LocalSecondaryIndexes { get; set; }
public Globalsecondaryindex[] GlobalSecondaryIndexes { get; set; }
public string BillingMode { get; set; }
public Provisionedthroughput ProvisionedThroughput { get; set; }
public Streamspecification StreamSpecification { get; set; }
public Ssespecification SSESpecification { get; set; }
public Tag[] Tags { get; set; }
}
public class Provisionedthroughput
{
public int ReadCapacityUnits { get; set; }
public int WriteCapacityUnits { get; set; }
}
public class Streamspecification
{
public bool StreamEnabled { get; set; }
public string StreamViewType { get; set; }
}
public class Ssespecification
{
public bool Enabled { get; set; }
public string SSEType { get; set; }
public string KMSMasterKeyId { get; set; }
}
public class Attributedefinition
{
public string AttributeName { get; set; }
public string AttributeType { get; set; }
}
public class Keyschema
{
public string AttributeName { get; set; }
public string KeyType { get; set; }
}
public class Localsecondaryindex
{
public string IndexName { get; set; }
public Keyschema1[] KeySchema { get; set; }
public Projection Projection { get; set; }
}
public class Projection
{
public string ProjectionType { get; set; }
public string[] NonKeyAttributes { get; set; }
}
public class Keyschema1
{
public string AttributeName { get; set; }
public string KeyType { get; set; }
}
public class Globalsecondaryindex
{
public string IndexName { get; set; }
public Keyschema2[] KeySchema { get; set; }
public Projection1 Projection { get; set; }
public Provisionedthroughput1 ProvisionedThroughput { get; set; }
}
public class Projection1
{
public string ProjectionType { get; set; }
public string[] NonKeyAttributes { get; set; }
}
public class Provisionedthroughput1
{
public int ReadCapacityUnits { get; set; }
public int WriteCapacityUnits { get; set; }
}
public class Keyschema2
{
public string AttributeName { get; set; }
public string KeyType { get; set; }
}
public class Tag
{
public string Key { get; set; }
public string Value { get; set; }
}