无法使用CosmosDB V3 SDK将cosmosdb容器的吞吐量从手动更改为自动缩放

时间:2020-08-18 09:27:45

标签: .net azure-cosmosdb azure-cosmosdb-sqlapi azure-cosmosdb-v3

我正在尝试使用此代码替换手动容器的吞吐量以自动缩放

container.ReplaceThroughputAsync(ThroughputProperties.CreateAutoscaleThroughput(4000));

这将引发异常。 错误”:[[必须提供x-ms-cosmos-migrate-off-to-autopilot,并且提供内容不得包含用于从手动吞吐量迁移到自动缩放的autopilotSettings。”

在CosmosDB文档中找不到与此相关的任何内容。我目前正在使用CosmosDB 3.12 V3 .Net SDK。

3 个答案:

答案 0 :(得分:2)

Azure Cosmos DB "Replace an Offer" REST API method允许在手动和自动缩放之间切换吞吐量模式。在这里,我不会重现该方法的文档的全部内容,但是要点是,根据要走的方向,您必须在正文的“ content”属性中提供一个特殊值,以及一个特定的自定义HTTP请求标头。

  • 手动->自动缩放
    • 内容:{ "offerThroughput": -1 }
    • 标题:x-ms-cosmos-migrate-offer-to-autopilot=true
  • 自动缩放->手动
    • 内容:{ "offerAutopilotSettings": {"maxThroughput": -1} }
    • 标题:x-ms-cosmos-migrate-offer-to-manual-throughput=true

使用REST API比SDK复杂得多。以下C#类总结了更改吞吐量方法。您将需要nuget包“ Microsoft.Azure.Cosmos”。

using System;
using System.Net;
using System.Net.Http;
using System.Reflection;
using System.Threading.Tasks;
using Microsoft.Azure.Cosmos;
using Newtonsoft.Json;

namespace ThroughputChangerDemo
{
    public class ThroughputChanger
    {
        private readonly HttpClient HttpClient;

        public ThroughputChanger(HttpClient httpClient)
        {
            HttpClient = httpClient;
        }

        public async Task SetThroughput(Database database, bool autoScale, int maxThroughput)
        {
            ThroughputResponse oldThroughputResponse = await database.ReadThroughputAsync(new RequestOptions { });
            var currentThroughput = oldThroughputResponse.Resource;

            ThroughputProperties newThroughput = GenerateThroughputProperties(autoScale, maxThroughput);

            if (currentThroughput.IsAutoScale() != autoScale)
            {
                await this.ChangeScalingMethodology(database.Client, currentThroughput, GenerateDatabaseLink(database));
            }

            await database.ReplaceThroughputAsync(newThroughput);
        }

        public async Task SetThroughput(Container container, bool autoScale, int maxThroughput)
        {
            ThroughputResponse oldThroughputResponse = await container.ReadThroughputAsync(new RequestOptions { });
            var currentThroughput = oldThroughputResponse.Resource;

            ThroughputProperties newThroughput = GenerateThroughputProperties(autoScale, maxThroughput);

            if (currentThroughput.IsAutoScale() != autoScale)
            {
                await this.ChangeScalingMethodology(container.Database.Client, currentThroughput, GenerateContainerLink(container));
            }

            await container.ReplaceThroughputAsync(newThroughput);
        }

        /// <summary>
        /// Toggle between Autoscale and Manual scaling methodologies for a database or container.
        /// </summary>
        /// <param name="currentThroughput"></param>
        /// <param name="scalableItemLink">The resource link for the database or container to be changed</param>
        /// <returns></returns>
        private async Task ChangeScalingMethodology(CosmosClient client, ThroughputProperties currentThroughput,
            string scalableItemLink)
        {
            bool changeToAutoScale = !currentThroughput.IsAutoScale();

            // Attempt to change between scaling schemes...
            string offerId = currentThroughput.SelfLink.Split('/')[1];
            string offerResource = $"offers/{offerId}";
            var url = $"{client.Endpoint.Scheme}://{client.Endpoint.Host}/{offerResource}";
            var restEndpointUri = new Uri(url);
            var method = HttpMethod.Put;
            var httpDate = DateTime.UtcNow.ToString("R");
            string auth = GenerateAuthToken(method, "offers", offerId,
                httpDate, extractAuthKey());
            var request = new HttpRequestMessage
            {
                RequestUri = restEndpointUri,
                Method = method,
                Headers = {
                    { HttpRequestHeader.Authorization.ToString(), auth },
                    { "x-ms-version", "2018-12-31" },
                    { "x-ms-date", httpDate },
                },
                Content = new StringContent(JsonConvert.SerializeObject(createOffer()))
            };

            if (changeToAutoScale)
            {
                request.Headers.Add("x-ms-cosmos-migrate-offer-to-autopilot", "true");
            }
            else
            {
                request.Headers.Add("x-ms-cosmos-migrate-offer-to-manual-throughput", "true");
            }

            HttpResponseMessage putResponse = await HttpClient.SendAsync(request);
            if (!putResponse.IsSuccessStatusCode)
            {
                var content = await putResponse.Content.ReadAsStringAsync();
                throw new Exception($"Error changing throughput scheme: '{putResponse.ReasonPhrase}'.\nContent: {content}");
            }

            // local function
            object createOffer()
            {
                // Read the ResourceRID using reflection because the property is protected.
                string resourceRID = currentThroughput.GetType()
                    .GetProperty("ResourceRID", BindingFlags.NonPublic | BindingFlags.Instance)
                    .GetValue(currentThroughput).ToString();
                string resourceLink = scalableItemLink;
                object content;
                if (changeToAutoScale)
                {
                    content = new
                    {
                        offerThroughput = -1
                    };
                }
                else
                {
                    content = new
                    {
                        offerAutopilotSettings = new { maxThroughput = -1 }
                    };
                }

                return new
                {
                    offerVersion = "V2",
                    offerType = "Invalid",
                    content = content,
                    resource = resourceLink,
                    offerResourceId = resourceRID,
                    id = offerId,
                    _rid = offerId,
                };
            }

            string extractAuthKey()
            {
                // Read the AccountKey using reflection because the property is protected.
                return client.GetType()
                    .GetProperty("AccountKey", BindingFlags.NonPublic | BindingFlags.Instance)
                    .GetValue(client).ToString();
            }
        }

        private string GenerateDatabaseLink(Database database) => $"dbs/{database.Id}/";

        private string GenerateContainerLink(Container container) => $"dbs/{container.Database.Id}/colls/{container.Id}/";

        private static ThroughputProperties GenerateThroughputProperties(bool autoScale, int? maxThroughput = null)
        {
            if (!autoScale)
            {
                if (!maxThroughput.HasValue || maxThroughput < 400)
                    maxThroughput = 400;
                return ThroughputProperties.CreateManualThroughput(maxThroughput.Value);
            }
            else
            {
                if (!maxThroughput.HasValue || maxThroughput < 4000)
                    maxThroughput = 4000;
                return ThroughputProperties.CreateAutoscaleThroughput(maxThroughput.Value);
            }
        }

        /// <summary>
        /// Generate the HTTP authorization header value needed to connect with Cosmos DB
        /// </summary>
        /// <param name="method">The Verb portion of the string is the HTTP verb, such as GET, POST, or PUT.</param>
        /// <param name="resourceType">The ResourceType portion of the string identifies the type of resource that the request is for, Eg. "dbs", "colls", "docs".</param>
        /// <param name="resourceLink">The ResourceLink portion of the string is the identity property of the resource that the request is directed at. ResourceLink must maintain its case for the ID of the resource. Example, for a container it looks like: "dbs/MyDatabase/colls/MyContainer".</param>
        /// <param name="date">The Date portion of the string is the UTC date and time the message was sent (in "HTTP-date" format as defined by RFC 7231 Date/Time Formats), for example, Tue, 01 Nov 1994 08:12:31 GMT. In C#, it can be obtained by using the "R" format specifier on the DateTime.UtcNow value. This same date(in same format) also needs to be passed as x-ms-date header in the request.</param>
        /// <param name="key">Cosmos DB key token (found in the Azure Portal)</param>
        /// <param name="keyType">denotes the type of token: master or resource.</param>
        /// <param name="tokenVersion">denotes the version of the token, currently 1.0.</param>
        /// <returns></returns>
        // This method borrowed from: https://docs.microsoft.com/en-us/rest/api/cosmos-db/access-control-on-cosmosdb-resources
        private string GenerateAuthToken(HttpMethod method, string resourceType, string resourceLink,
            string date, string key, string keyType = "master", string tokenVersion = "1.0")
        {
            var hmacSha256 = new System.Security.Cryptography.HMACSHA256 { Key = Convert.FromBase64String(key) };

            var verb = method?.Method ?? "";
            resourceType = resourceType ?? "";
            resourceLink = resourceLink?.ToLower() ?? ""; // Without ToLower(), we get an 'unauthorized' error.

            string payLoad = string.Format(System.Globalization.CultureInfo.InvariantCulture, "{0}\n{1}\n{2}\n{3}\n{4}\n",
                verb.ToLowerInvariant(),
                resourceType.ToLowerInvariant(),
                resourceLink,
                date.ToLowerInvariant(),
                ""
            );

            byte[] hashPayLoad = hmacSha256.ComputeHash(System.Text.Encoding.UTF8.GetBytes(payLoad));
            string signature = Convert.ToBase64String(hashPayLoad);

            return System.Web.HttpUtility.UrlEncode(String.Format(System.Globalization.CultureInfo.InvariantCulture, "type={0}&ver={1}&sig={2}",
                keyType,
                tokenVersion,
                signature));
        }
    }
}

用法:

var httpClient = new HttpClient();
var cosmosClient = new CosmosClient(EndpointUrl, PrimaryKey);
var database = cosmosClient.GetDatabase(DatabaseId);
var changer = new ThroughputChanger(httpClient);
await changer.SetThroughput(database, autoScale: true, maxThroughput: 8000);
httpClient.Dispose();

答案 1 :(得分:0)

现在不支持通过sdk将吞吐量从手动缩放更改为自动缩放。方法ReplaceThroughputAsync仅可以更改吞吐量。您应该在Azure门户上进行更改。

答案 2 :(得分:0)

注意:不是这个问题的答案,而是与cosmos-db数据库处于自动缩放模式时以编程方式更改吞吐量有关。

使用Microsoft.Azure.Cosmos;

 public async Task<ThroughputResponse> UpdateThroughput(int targetLevel)
    {
        Container container = cosmosClient.GetContainer(databaseName, collectionName);
        ThroughputResponse throughput = await container.ReplaceThroughputAsync(ThroughputProperties.CreateAutoscaleThroughput(targetLevel));            
        return throughput;
    }

用例:我必须每晚进行一次工作,每小时的工作量要求很高(50K RU / s),但我的正常负载不高于(10K RU / s)。在开始这项工作时,我将自动缩放比例提高到50K,在完成工作后,我将其降低了10K以优化成本。由于费用从(x的10%)到x不等,所以我想维持一个成本最优的阈值。

进一步阅读:https://github.com/MicrosoftDocs/azure-docs/blob/master/articles/cosmos-db/how-to-provision-autoscale-throughput.md#change-the-autoscale-max-throughput-rus

相关问题