Golang Mgo起搏

Question

我正在编写一个快速写入mongodb的应用程序。对于mongodb和mgo来说太快了。我的问题是，有没有办法让我确定mongo无法跟上并开始阻止？但我也不想不必要地阻止。 以下是模拟问题的代码示例：

package main

import (
  "labix.org/v2/mgo"
  "time"
  "fmt"
)

// in database name is a string and age is an int

type Dog struct{
  Breed string "breed"
}

type Person struct{
  Name string "name"
  Pet Dog `bson:",inline"`
  Ts        time.Time
}

func insert(session *mgo.Session, bob Person){
  err := session.DB("db_log").C("people").Insert(&bob)
  if err != nil {
    panic("Could not insert into database")
  }
}

func main() {
  session, _ := mgo.Dial("localhost:27017")
  bob := Person{Name : "Robert", Pet : Dog{}}
  i := 0
  for {
    time.Sleep(time.Duration(1) * time.Microsecond)
    i++
    go insert(session, bob)
  }
}

我经常会遇到如下错误：

panic: Could not insert into database

或

panic: write tcp 127.0.0.1:27017: i/o timeout

Answer 1

如果您allow Go to use multiple threads和Copy() then Close()您的会话，我怀疑您会获得更好的表现。

要回答你的问题，这可能是一个完美的频道使用案例。将项目输入到一个goroutine中的通道中并将它们消耗/写入另一个中的Mongo。您可以调整通道的大小以满足您的需求。生成器线程在尝试发送到它时，一旦通道已满就会阻塞。

您可能还想使用Safe()方法设置。设置W：0将使Mongo处于“即发即忘”模式，这将大大加快性能，但可能会丢失一些数据。您也可以更改超时时间。

Answer 2

我还没有测试过，但我认为这段代码应该可行。 在长时间保持会话后我得到了这个问题，这样我就可以每隔一定时间更新一次会话。

package main

import (
  "gopkg.in/mgo.v2"
  "time"
  "fmt"
)

// in database name is a string and age is an int

type Dog struct{
  Breed string "breed"
}

type Person struct{
  Name string "name"
  Pet Dog `bson:",inline"`
  Ts        time.Time
}

func insert(session *mgo.Session, bob Person){
  err := session.DB("db_log").C("people").Insert(&bob)
  if err != nil {
    panic("Could not insert into database")
  }
}

func main() {
  current_session, _ := mgo.Dial("localhost:27017")
  using_session := current_session
  bob := Person{Name : "Robert", Pet : Dog{}}

  /*
  * this technical to prevent connect timeout after long time connection on mongodb from golang session
  * Idea is simple: the session will be renew after certain time such as 1 hour
  */
  //ticker := time.NewTicker(time.Hour * 1)

  //Set 10 seconds for test
  ticker := time.NewTicker(time.Second * 10)

  go func() {

    for t := range ticker.C {
      fmt.Println("Tick at", t)
      new_session := current_session.Copy()
      fmt.Printf("Current session here %p\n", current_session)
      fmt.Printf("New session here %p\n", new_session)
      using_session = new_session
      //setTimeout 30 second before close old sesion, to make sure current instance use current connection isn't affect
      //time.AfterFunc(time.Second * 30, func() { 

      //Set 2 seconds for test
      time.AfterFunc(time.Second * 2, func() { 

        //close previous session

        current_session.Close()
        current_session = new_session

        //assign to new session

      })

    }
  }()

  i := 0
  for {
    time.Sleep(time.Duration(1) * time.Microsecond)
    i++
    go insert(using_session, bob)
  }

}