Skip to main content

消息系统设计要点

操作系统
消息传递作为进程间通信的重要一种，他提供了网络间进程通信的方式。但是相比于管程和信号量等，他具有不稳定性。
需要考的问题简单来说有这么几点

  1. 消息传递的不可靠性解决。在网络间传递时，A到B已经发送了消息，B可能没有收到，A要有一种机制可以知道B已经收到了消息。因此，B需要发送一个ACknowledge给A，来保证他已经收到。
  2. 消息传递的重复性。在A重复的发送了数据后，如果经过一定的时间后，A没有收到ACK，所以又重复发了一次相同的message，但是经过一段时间后，B收到了两个相同消息，如何进行区分？可以在发送的消息中加入连续的序列来表达不同的消息。
  3. 还有许多其他问题，这里直说两个比较普遍的问题。

Comments

Post a Comment

Popular posts from this blog

Elasticsearch error when the field exceed limit 32kb

When we post data that the field exceed the limit, elasticsearch will reject the data which error: {"error":{"root_cause":[{"type":"remote_transport_exception","reason":"[cs19-2][10.200.20.39:9301][indices:data/write/index]"}],"type":"illegal_argument_exception","reason":"Document contains at least one immense term in field=\"field1\" (whose UTF8 encoding is longer than the max length 32766), all of which were skipped.  Please correct the analyzer to not produce such terms.  The prefix of the first immense term is You can handle this: 1. you can udpate the mapping part at any time PUT INDEXNAME/_mapping/TYPENAME { "properties" : { "logInfo" : { "type" : "string" , "analyzer" : "keyword" , "ignore_above" : 32766 } } } ignore_above means that we will only keep 32766 bytes 2...
Post a Comment
Read more

Scrapy ERROR :ImportError: Error loading object 'scrapy.telnet.TelnetConsole': No module named conch

原文: https://stackoverflow.com/questions/17263509/error-while-using-scrapy-scrapy-telnet-telnetconsole-no-module-named-conch/17264705#17264705 On Ubuntu, you should avoid using  easy_install  wherever you can. Instead, you should be using  apt-get ,  aptitude , "Ubuntu Software Center", or another of the distribution-provided tools. For example, this single command is all you need to install scrapy - along with every one of its dependencies that is not already installed: $ sudo apt - get install python - scrapy easy_install  is not nearly as good at installing things as  apt-get . Chances are the reason you can't get it to work is that it didn't quite install things sensibly, particularly with respect to what was already installed on the system. Sadly, it also leaves no record of what it did, so uninstallation is difficult or impossible. You may now have a big mess on your system that prevents proper installations from working as well (or maybe not...
Post a Comment
Read more

Golang http server performance tuning practice (Golang HTTP服务器性能调优实践)

  Golang 1.8后本身自带了pprof的这个神器，可以帮助我们很方便的对服务做一个比较全面的profile。对于一个Golang的程序，可以从多个方面进行profile，比如memory和CPU两个最基本的指标，也是我们最关注的，同时对特有的goroutine也有运行状况profile。关于golang profiling本身就不做过多的说明，可以从 官方博客 中了解到详细的过程。   Profile的环境 Ubuntu 14.04.4 LTS (GNU/Linux 3.19.0-25-generic x86_64) go version go1.9.2 linux/amd64  profile的为一个elassticsearch数据导入接口，承担接受上游数据，根据元数据信息写入相应的es索引中。目前的状况是平均每秒是1.3Million的Doc数量。   在增加了profile后，从CPU层面发现几个问题。 runtime mallocgc 占用了17.96%的CPU。 SVG部分图如下 通过SVG图，可以看到调用链为： ioutil.ReadAll -> buffer.ReadFrom -> makeSlice -> malloc.go  然后进入ReadAll的源码。 readAll()方法 func readAll(r io.Reader, capacity int64) (b []byte, err error) { buf := bytes . NewBuffer ( make ([] byte , 0 , capacity )) // If the buffer overflows, we will get bytes.ErrTooLarge. // Return that as an error. Any other panic remains.   defer func() { e := recover () if e == nil { return } if panicErr , ok := e .( error ) ; ok && p...
Post a Comment
Read more