concurrency - Go concurrent slice access -
i'm doing stream processing in go , got stuck trying figure out how "go way" without locks.
this contrived example shows problem i'm facing.
- we 1
thing@ time. - there goroutine buffers them slice called
things. - when
thingsbecomes fulllen(things) == 100processed somehow , reset - there
nnumber of concurrent goroutines need accessthingsbefore it's full - access "incomplete"
thingsother goroutines not predictable. - neither
dosomethingwithpartialnordosomethingwithcompleteneeds mutatethings
code:
var m sync.mutex var count int64 things := make([]int64, 0, 100) // slices of data being generated , used go func() { { m.lock() if len(things) == 100 { // dosomethingwithcomplete not modify things dosomethingwithcomplete(things) things = make([]int64, 0, 100) } things = append(things, count) m.unlock() count++ } }() // dosomethingwithpartial needs access things before they're ready { m.lock() // dosomethingwithpartial not modify things dosomethingwithpartial(things) m.unlock() } i know slices immutable mean can remove mutex , expect still work (i assume no).how can refactor use channels instead of mutex.
edit: here's solution came not use mutex
package main import ( "fmt" "sync" "time" ) func incrementor() chan int { ch := make(chan int) go func() { count := 0 { ch <- count count++ } }() return ch } type foo struct { things []int requests chan chan []int stream chan int c chan []int } func newfoo() *foo { foo := &foo{ things: make([]int, 0, 100), requests: make(chan chan []int), stream: incrementor(), c: make(chan []int), } go foo.launch() return foo } func (f *foo) launch() { { select { case ch := <-f.requests: ch <- f.things case thing := <-f.stream: if len(f.things) == 100 { f.c <- f.things f.things = make([]int, 0, 100) } f.things = append(f.things, thing) } } } func (f *foo) things() []int { ch := make(chan []int) f.requests <- ch return <-ch } func main() { foo := newfoo() var wg sync.waitgroup wg.add(10) := 0; < 10; i++ { go func(i int) { time.sleep(time.millisecond * time.duration(i) * 100) things := foo.things() fmt.println("got things:", len(things)) wg.done() }(i) } go func() { _ = range foo.c { // things } }() wg.wait() }
it should noted "go way" use mutex this. it's fun work out how channel mutex simpler , easier reason particular problem.
Comments
Post a Comment