Go Concurrency: Difference between revisions

From NovaOrdis Knowledge Base
Jump to navigation Jump to search
 
(43 intermediate revisions by the same user not shown)
Line 2: Line 2:
* [[Go_Language#Concurrency|Go Language]]
* [[Go_Language#Concurrency|Go Language]]
* [[Concurrent_(Parallel)_Programming|Concurrent (Parallel) Programming]]
* [[Concurrent_(Parallel)_Programming|Concurrent (Parallel) Programming]]
=TODO=
<font color=darkkhaki>
* Recursive Locking
** Resolve the problem of reentrant locks and come up with a valid pattern. Update [[Go_Mutex_and_RWMutex#Reentrance|Go Mutex and RWMutex &#124; Reentrance]]
** https://stackoverflow.com/questions/14670979/recursive-locking-in-go
** https://rationaleemotions.com/reentrancy_not_permitted/
* https://go.dev/talks/2013/advconc.slide#1 + http://www.youtube.com/watch?v=QDDwwePbDtw
* https://go.dev/blog/pipelines
* https://go.dev/talks/2012/concurrency.slide#1 + https://www.youtube.com/watch?v=f6kdp27TYZs
* https://medium.com/insiderengineering/concurrent-http-requests-in-golang-best-practices-and-techniques-f667e5a19dea
* https://blogtitle.github.io/categories/concurrency/
* Rethinking Classical Concurrency Patterns https://drive.google.com/file/d/1nPdvhB0PutEJzdCq5ms6UI58dp50fcAN/view?pli=1
* https://medium.com/@souravchoudhary0306/healing-unhealthy-goroutines-in-long-running-programs-b2a2b56976e5
</font>


=Overview=
=Overview=


The majority of programming languages address concurrency by providing a representation of [[Concurrent_(Parallel)_Programming#O/S_Threads|O/S threads]] or [[Concurrent_(Parallel)_Programming#Green_Threads|green threads]] directly in the language or in libraries, and providing [[Concurrent_(Parallel)_Programming#Memory_Access_Synchronization_Primitives|memory access synchronization primitives]] to protect data in presence of concurrent access.
The majority of programming languages address concurrency by providing a representation of [[Concurrent_(Parallel)_Programming#O/S_Threads|O/S threads]] or [[Concurrent_(Parallel)_Programming#Green_Threads|green threads]] directly in the language or in libraries, and exposing [[Concurrent_(Parallel)_Programming#Memory_Access_Synchronization_Primitives|memory access synchronization primitives]] to protect data in presence of concurrent access.


Go takes a different approach. The fundamental concurrent execution primitive is the [[Go_Language_Goroutines#Overview|goroutine]], exposed by the language with the [[Go_Language#Keywords|language keyword]] <code>[[Go_Language#go_keyword|go]]</code>. Goroutines are managed by the Go runtime, which maps them transparently onto threads. While Go provides memory access and thread synchronization primitives at the language level via the <code>[[Go_Package_sync|sync]]</code> package, language guidelines do not exactly encourage their use, unless in very specific situations, such as within a small <code>struct</code> scope. Go provides an alternative concurrency programming model in language, based on [[Concurrent_(Parallel)_Programming#Communicating_Sequential_Processes_(CSP)|Communicating Sequential Processes (CSP)]], a theoretical model introduced by Anthony Hoare in the "Communicating Sequential Processes" 1978 ACM paper. This concurrency programming model uses [[Go Language Channels#Overview|channels]], which are exposed at language level with the <code>[[Go_Language#chan_keyword|chan]]</code> keyword.
Go takes a different approach.  


This approach allows you to directly map concurrent problems into concurrent constructs instead of dealing with the minutia of starting and managing threads, and mapping logic evenly across available threads. In Go, programmers model concurrency in goroutines, as concurrent functions, and channels, and communication mechanisms between parallel functions. In languages that expose threads, you need to worry about things like thread pools, and map requests onto threads. In contrast, in Go you would write a function and prepend its invocation with the <code>go</code> keyword. The runtime handles everything else automatically.
The fundamental concurrent execution primitive is the [[Go_Language_Goroutines#Overview|goroutine]], introduced in the language by the [[Go_Language#Keywords|keyword]] <code>[[Go_Language#go_keyword|go]]</code>. Goroutines are managed by the [[Go_Runtime#Overview|Go runtime]], which maps them transparently onto threads. The O/S threads are not exposed in the concurrency programming model.
 
While Go provides [[Go_Package_sync#Memory_Access_Synchronization_Primitives|memory access]] and [[Go_Package_sync#Thread_Synchronization_Primitives|thread synchronization primitives]] at the language level via the <code>[[Go_Package_sync|sync]]</code> package, the language guidelines do not exactly encourage their use, unless in very specific situations, such as within a [[Go_Mutex_and_RWMutex#Guarding_the_Internal_State_of_a_struct|small <code>struct</code> scope]]. Go provides an alternative concurrency programming model in the language, based on [[Concurrent_(Parallel)_Programming#Communicating_Sequential_Processes_(CSP)|Communicating Sequential Processes (CSP)]], a theoretical model introduced by Anthony Hoare in the "Communicating Sequential Processes" 1978 ACM paper. This concurrency programming model uses [[Go_Channels#Overview|channels]], which are introduced in the language by the <code>[[Go_Language#chan_keyword|chan]]</code> keyword.
 
This approach allows you to directly map concurrent problems onto concurrent constructs instead of dealing with the minutia of starting and managing threads, and mapping logic evenly across available threads. In Go, programmers model concurrency as goroutines and channels. In languages that expose threads, you need to worry about things like thread pools, and map requests onto threads. In contrast, in Go you would write a function and prepend its invocation with the <code>go</code> keyword. The runtime handles everything else automatically.
 
Go's '''philosophy on concurrency''' can be summed up like this: aim for simplicity, use channels when possible, and treat goroutines as a free resource.


=Goroutines=
=Goroutines=
{{Internal|Go_Language_Goroutines#Overview|Goroutines}}
{{Internal|Go_Language_Goroutines#Overview|Goroutines}}
=Programming Models=
=Programming Models=
Go provides two main programming models when it comes to modeling concurrency in the program. The preferred way is based on communicating sequential processes (CSP) paradigm and encourages the use of channels. Memory access synchronization is available, but the language designers [[Go_Package_sync#Overview|advise against using it as the first choice]]. This is a workflow to help selecting one model versus the other.
Go provides two main programming models for concurrency. The preferred one is based on communicating sequential processes (CSP) paradigm and encourages the use of [[Go_Channels#Overview|channels]]. Memory access synchronization is available, but the language designers [[Go_Package_sync#Overview|advise against using it as the first choice]]. This is a workflow to help selecting one model versus the other.


<font size=-2>
<font size=-2>
Line 23: Line 43:
       │                              │
       │                              │
       │                              ▼                        YES <span id='Transferring_the_Ownership_of_Data'></span>
       │                              ▼                        YES <span id='Transferring_the_Ownership_of_Data'></span>
       │        Are you trying to [[Go_Language_Channels#Transferring_the_Ownership_of_Data|transfer ownership of data?]] ──────────────┐
       │        Are you trying to [[Go_Channels#Transferring_the_Ownership_of_Data|transfer ownership of data?]] ──────────────┐
       │                              │ NO                                  │
       │                              │ NO                                  │
       │                              │                                      │
       │                              │                                      │
Line 30: Line 50:
       │                              │ NO                                  │
       │                              │ NO                                  │
       │                              │                                      │
       │                              │                                      │
       │  NO                        ▼                              YES    
       │  NO                        ▼                              YES <span id='Coordinate_Multiple_Pieces_of_Logic'></span>   
       ├────── Are you trying to coordinate multiple pieces of logic? ───────┤
       ├────── Are you trying to [[Go_Channels#Coordinate_Multiple_Pieces_of_Logic|coordinate multiple pieces of logic?]] ───────┤
       │                                                                    │
       │                                                                    │
       ▼                                                                    ▼
       ▼                                                                    ▼
   Use [[Go_Package_sync#Overview|Primitives]]                                                        Use [[Go_Language_Channels#Overview|Channels]]
   Use [[Go_Package_sync#Overview|Primitives]]                                                        Use [[Go_Channels#Overview|Channels]]
   
   
   
   
Line 40: Line 60:


==Channels==
==Channels==
{{Internal|Go Language Channels#Overview|Channels}}
{{Internal|Go_Channels#Overview|Channels}}


==<span id='hMemory_Access_and_Thread_Synchronization_Primitives'></span>Synchronization Primitives==
==<span id='hMemory_Access_and_Thread_Synchronization_Primitives'></span>Synchronization Primitives==
Line 46: Line 66:
===Memory Access Synchronization Primitives===
===Memory Access Synchronization Primitives===
{{Internal|Go_Package_sync#Memory_Access_Synchronization_Primitives|Memory Access Synchronization Primitives}}
{{Internal|Go_Package_sync#Memory_Access_Synchronization_Primitives|Memory Access Synchronization Primitives}}
===Thread Synchronization Primitives===
===<span id='Thread_Synchronization_Primitives'></span>Execution Synchronization Primitives===
{{Internal|Go_Package_sync#Thread_Synchronization_Primitives|Thread Synchronization Primitives}}
{{Internal|Go_Package_sync#Thread_Synchronization_Primitives|Execution Synchronization Primitives}}
 
=Context=
{{Internal|Go Context#Overview|Context}}


=Patterns=
=<span id='Patterns'></span>Concurrency Patterns=
{{Internal|Go_Language_Channels#Channel_Patterns|Channel Patterns}}
* [[Go_Once#Initialization_Pattern|Initialization pattern with <tt>sync.Once</tt>]]
* <span id='context'></span><span id='Context'></span>[[Go_Package_context#Programming_Model|<tt>context.Context</tt>]]
* [[Go_Channels#Channel_Patterns|Channel Patterns]]
* [[Go_Language_Goroutines#Goroutine_Patterns|Goroutine Patterns]]
* [[Go Heartbeat#Overview|Heartbeat]]
* [[Go Replicating Requests#Overview|Replicating Requests]]
* [[Go Rate Limiting#Overview|Rate Limiting]]

Latest revision as of 20:47, 14 May 2024

Internal

TODO

Overview

The majority of programming languages address concurrency by providing a representation of O/S threads or green threads directly in the language or in libraries, and exposing memory access synchronization primitives to protect data in presence of concurrent access.

Go takes a different approach.

The fundamental concurrent execution primitive is the goroutine, introduced in the language by the keyword go. Goroutines are managed by the Go runtime, which maps them transparently onto threads. The O/S threads are not exposed in the concurrency programming model.

While Go provides memory access and thread synchronization primitives at the language level via the sync package, the language guidelines do not exactly encourage their use, unless in very specific situations, such as within a small struct scope. Go provides an alternative concurrency programming model in the language, based on Communicating Sequential Processes (CSP), a theoretical model introduced by Anthony Hoare in the "Communicating Sequential Processes" 1978 ACM paper. This concurrency programming model uses channels, which are introduced in the language by the chan keyword.

This approach allows you to directly map concurrent problems onto concurrent constructs instead of dealing with the minutia of starting and managing threads, and mapping logic evenly across available threads. In Go, programmers model concurrency as goroutines and channels. In languages that expose threads, you need to worry about things like thread pools, and map requests onto threads. In contrast, in Go you would write a function and prepend its invocation with the go keyword. The runtime handles everything else automatically.

Go's philosophy on concurrency can be summed up like this: aim for simplicity, use channels when possible, and treat goroutines as a free resource.

Goroutines

Goroutines

Programming Models

Go provides two main programming models for concurrency. The preferred one is based on communicating sequential processes (CSP) paradigm and encourages the use of channels. Memory access synchronization is available, but the language designers advise against using it as the first choice. This is a workflow to help selecting one model versus the other.

               YES
     ┌──────────── Is it a performance critical section? 
     │                              │ NO
     │                              │
     │                              ▼                        YES 
     │         Are you trying to transfer ownership of data? ──────────────┐
     │                              │ NO                                   │
     │                              │                                      │
     │   YES                        ▼                                      │ 
     ├────── Are you trying to guard the internal state of a struct?       │
     │                              │ NO                                   │
     │                              │                                      │
     │   NO                         ▼                              YES     │
     ├────── Are you trying to coordinate multiple pieces of logic? ───────┤
     │                                                                     │
     ▼                                                                     ▼
 Use Primitives                                                        Use Channels


Channels

Channels

Synchronization Primitives

The sync package provides memory access and thread synchronization primitives.

Memory Access Synchronization Primitives

Memory Access Synchronization Primitives

Execution Synchronization Primitives

Execution Synchronization Primitives

Concurrency Patterns