使用K8S进行横向扩展

时间:2018-08-20 13:36:30

标签: kubernetes

我们有一个使用基于Java的HTTP服务器的pod。 HTTP POST请求从JMeter发送到服务器。我们目前正在通过增加副本数来测量此http服务器的横向扩展百分比。此处附带的是部署yaml,它配置为使用nodePort和hostPort公开服务。

我们的测试显示nodePort横向扩展百分比为57,而hostPort横向扩展百分比为〜95。

我们想知道为什么这种行为上的差异以及如何调整nodePort以获得与hostPort相同的横向扩展百分比。

以下是测试详细信息:

  

K8S版本: v1.9.6,

     

系统详细信息::1个主服务器,3个工作器,1个NFS服务器

     

操作系统:CentOS Linux版本7.3

     

系统资源:16个GiB RAM(在每个K8S节点中),12个CPU(在每个K8S中)   节点),1 TB的存储空间。

Deployment YAML file

apiVersion: v1
kind: Service
metadata:
  name: itom-di-receiver-svc
  namespace: opsbridge1
spec:
  type: NodePort
  ports:
  - name: receiver-port
    nodePort: 30001
    port: 8443
    protocol: TCP
    targetPort: 5050
  selector:
    app: itom-di-receiver

---

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: itom-di-receiver-dpl
  namespace: opsbridge1
  annotations:
    deployment.microfocus.com/default-replica-count: "1"
    deployment.microfocus.com/runlevel: UP
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: itom-di-receiver
      annotations:
        pod.boostport.com/vault-approle: opsbridge1-di
        pod.boostport.com/vault-init-container: install
    spec:
      containers:
        - name: itom-di-receiver-cnt
          image: localhost:5000/hpeswitomsandbox/itom-data-ingestion-receiver:1.3.0-029
          livenessProbe:
            exec:
              command:
              - cat
              - /receiver/receiver-status.txt
            initialDelaySeconds: 180
            periodSeconds: 20
          readinessProbe:
            exec:
              command:
              - "/receiver/bin/readiness.sh"
            initialDelaySeconds: 30
            periodSeconds: 20
            failureThreshold: 18
          securityContext:
            capabilities:
              drop:
                - all
              add:
                - CHOWN
                - SETGID
                - SETUID
          imagePullPolicy: IfNotPresent
          env:
          - name: gid
            value: "1999"
          - name: uid
            value: "1999"
          - name: KAFKA_SECURITY_PROTOCOL
            value: "SSL"
          - name: KAFKA_HOSTNAME
            valueFrom:
              configMapKeyRef:
                name: itom-di-kafka-cm
                key: kafka.advertised.host.name
          - name: KAFKA_PORT
            valueFrom:
              configMapKeyRef:
                name: itom-di-kafka-cm
                key: kafka.advertised.port
          - name: KAFKA_DEFAULT_TOPIC
            valueFrom:
              configMapKeyRef:
                name: itom-di-receiver-cm
                key: kafka.default.topic
          - name: KAFKA_MAP_CONTEXT_TO_TOPIC
            valueFrom:
              configMapKeyRef:
                name: itom-di-receiver-cm
                key: kafka.map.context.to.topic
          - name: RECEIVER_KAFKA_TLS_ENABLE
            valueFrom:
              configMapKeyRef:
                name: itom-di-receiver-cm
                key: receiver.kafka.tls.enable
          - name: RECEIVER_JVM_ARGS
            valueFrom:
              configMapKeyRef:
                name: itom-di-receiver-cm
                key: receiver.jvm.args
           ### Receiver Performance Tuning Environment Variables ###
          - name: RECEIVER_COMPUTE_THREADS
            valueFrom:
              configMapKeyRef:
                name: itom-di-receiver-cm
                key: receiver.compute.threads
          - name: KAFKA_PRODUCER_BATCH_SIZE
            valueFrom:
              configMapKeyRef:
                name: itom-di-receiver-cm
                key: receiver.kafka.producer.batch.size.kb
          - name: RECEIVER_REQUEST_BODY_MAXSIZE_MB
            valueFrom:
              configMapKeyRef:
                name: itom-di-receiver-cm
                key: receiver.request.body.maxsize.mb
          - name: KAFKA_PRODUCER_LINGER_MS
            valueFrom:
              configMapKeyRef:
                name: itom-di-receiver-cm
                key: receiver.kafka.producer.linger.ms
          - name: KAFKA_PRODUCER_ACKS
            valueFrom:
              configMapKeyRef:
                name: itom-di-receiver-cm
                key: receiver.kafka.producer.acks
          - name: KAFKA_PRODUCER_COMPRESSION_TYPE
            valueFrom:
              configMapKeyRef:
                name: itom-di-receiver-cm
                key: receiver.kafka.producer.compression.type
          - name: KAFKA_PRODUCER_BUFFER_MEMORY
            valueFrom:
              configMapKeyRef:
                name: itom-di-receiver-cm
                key: receiver.kafka.producer.buffer.memory.mb
          - name: KAFKA_PRODUCER_MAX_BLOCK_MS
            valueFrom:
              configMapKeyRef:
                name: itom-di-receiver-cm
                key: receiver.kafka.producer.max.block.ms
          - name: RECEIVER_HEADER_FIELDNAME_FOR_TOPIC
            valueFrom:
              configMapKeyRef:
                name: itom-di-receiver-cm
                key: receiver.header.fieldname.for.topic
          - name: RECEIVER_HEADER_FIELDNAME_FOR_TOPIC_KEY
            valueFrom:
              configMapKeyRef:
                name: itom-di-receiver-cm
                key: receiver.header.fieldname.for.topic.key
          - name: RECEIVER_TOPIC_FROM_HEADER
            valueFrom:
              configMapKeyRef:
                name: itom-di-receiver-cm
                key: receiver.topic.from.header
          - name: KAFKA_PRODUCER_RETRIES
            valueFrom:
              configMapKeyRef:
                name: itom-di-receiver-cm
                key: receiver.kafka.producer.retries
          - name: KAFKA_PRODUCER_MAX_IN_FLIGHT_REQUEST_PER_CONNECTION
            valueFrom:
              configMapKeyRef:
                name: itom-di-receiver-cm
                key: receiver.kafka.producer.max.in.flight.requests.per.connection

          ### Security Environment Variables ###
          - name: RECEIVER_PROTOCOL
            valueFrom:
              configMapKeyRef:
                name: itom-di-receiver-cm
                key: receiver.protocol
          - name: RECEIVER_AUTH_METHOD
            valueFrom:
              configMapKeyRef:
                name: itom-di-receiver-cm
                key: receiver.auth.method
          - name: RECEIVER_KEYSTORE_TYPE
            valueFrom:
              configMapKeyRef:
                name: itom-di-receiver-cm
                key: receiver.keystore.type
          - name: RECEIVER_TRUSTSTORE_TYPE
            valueFrom:
              configMapKeyRef:
                name: itom-di-receiver-cm
                key: receiver.truststore.type
          - name: RECEIVER_EXTERNAL_JAR_ENABLE
            valueFrom:
              configMapKeyRef:
                name: itom-di-receiver-cm
                key: receiver.external.jar.enable
          - name: RECEIVER_JAR_VALIDATE
            valueFrom:
              configMapKeyRef:
                name: itom-di-receiver-cm
                key: receiver.jar.validate
          - name: CERTIFICATE_REVOCATION
            valueFrom:
              configMapKeyRef:
                name: itom-di-receiver-cm
                key: receiver.cert.revocation.enable
          - name: CRL_FILE
            valueFrom:
              configMapKeyRef:
                name: itom-di-receiver-cm
                key: receiver.cert.revocation.crlfile.enable
          - name: PREFER_OCSP
            valueFrom:
              configMapKeyRef:
                name: itom-di-receiver-cm
                key: receiver.cert.revocation.ocsp.enable
          - name: ENABLE_SOFT_FAIL
            valueFrom:
              configMapKeyRef:
                name: itom-di-receiver-cm
                key: receiver.cert.revocation.softfail.enable
          - name: PREFER_CRL
            valueFrom:
              configMapKeyRef:
                name: itom-di-receiver-cm
                key: receiver.cert.revocation.preferCRL.enable
          - name: RESPONDER_URL
            valueFrom:
              configMapKeyRef:
                name: itom-di-receiver-cm
                key: ocsp.responderURL
          - name: CERT_SUBJECT_NAME
            valueFrom:
              configMapKeyRef:
                name: itom-di-receiver-cm
                key: ocsp.responderCertSubjectName
          - name: CERT_ISSUER_NAME
            valueFrom:
              configMapKeyRef:
                name: itom-di-receiver-cm
                key: ocsp.responderCertIssuerName
          - name: CERT_SERIAL_NUMBER
            valueFrom:
              configMapKeyRef:
                name: itom-di-receiver-cm
                key: ocsp.responderCertSerialNumber
          - name: RECEIVER_FIPS_ENABLE
            valueFrom:
              configMapKeyRef:
                name: itom-di-receiver-cm
                key: receiver.fips.enable
          - name: RECEIVER_APIKEY_REFRESH_INTERVAL_MINS
            valueFrom:
              configMapKeyRef:
                name: itom-di-receiver-cm
                key: receiver.apikey.refresh.interval.mins
          ports:
          - containerPort: 5050
            hostPort: 5051
          resources:
            limits:
              cpu: "4"
              memory: "2048Mi"
            requests:
              cpu: "1"
              memory: "1024Mi"
          volumeMounts:
          - name: di-receiver-volume
            mountPath: /receiver/conf
            subPath: di/receiver/conf
          - name: di-receiver-volume
            mountPath: /receiver/conf/endpoint
            subPath: di/receiver/conf/endpoint
          - name: di-receiver-volume
            mountPath: /receiver/conf/schema
            subPath: di/receiver/conf/schema
          - name: di-receiver-volume
            mountPath: /receiver/conf/crl
            subPath: di/receiver/conf/crl
          - name: di-receiver-log-volume
            mountPath: /receiver/log
            subPath: di/receiver/log
          - name: di-receiver-volume
            mountPath: /receiver/ext
            subPath: di/receiver/ext
          - name: di-receiver-volume
            mountPath: /receiver/data
            subPath: di/receiver/data
          - name: di-receiver-volume
            mountPath: /receiver/samples
            subPath: di/receiver/samples
          - name: vault-token
            mountPath: /var/run/secrets/boostport.com
          - name: secret-volume
            mountPath: /receiver/ssl/ca
          - name: secret-volume-kafka
            mountPath: /receiver/ssl/store/receiver-kafka
        - name: kubernetes-vault-renew
          image: localhost:5000/kubernetes-vault-renew:0.5.0
          imagePullPolicy: IfNotPresent
          volumeMounts:
          - name: vault-token
            mountPath: /var/run/secrets/boostport.com
      initContainers:
      - env:
        - name: VAULT_ROLE_ID
          value: "66d8c1aa-6079-a65f-38c3-89bd7a6fdd2c"
        - name: CERT_COMMON_NAME
          value: "smperfqa31.hpeswlab.net"
        image: localhost:5000/kubernetes-vault-init:0.5.0
        imagePullPolicy: IfNotPresent
        name: install
        resources: {}
        securityContext:
          runAsUser: 1999
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
          - mountPath: /var/run/secrets/boostport.com
            name: vault-token
      - name: itom-di-init-receiver-cnt
        image: localhost:5000/hpeswitomsandbox/itom-data-ingestion-receiver:1.3.0-029
        command: ["/receiver/bin/run.sh","dependency"]
        env:
        - name: KAFKA_HOSTNAME
          valueFrom:
            configMapKeyRef:
              name: itom-di-kafka-cm
              key: kafka.advertised.host.name
      volumes:
        - name: di-receiver-volume
          persistentVolumeClaim:
             claimName: conf-volume
        - name: di-receiver-log-volume
          persistentVolumeClaim:
             claimName: log-volume
        - name: vault-token
          emptyDir: {}
        - name: secret-volume
          secret:
            secretName: receiver-secret
        - name: secret-volume-kafka
          secret:
            secretName: receiver-kafka-secret

0 个答案:

没有答案