When I create a mutex semaphore, is the initial condition such that a "Take" will block until a "Give" is issued, or is it necessary to issue a starting "Take" to establish the initial state?

In other words, if I issue a "Create" then a "Take" before any other reference to the semaphore, will the task block? It's not clear in the documentation about initial states, though the example implies the task will block.

I assumed the underlying queue is empty so a take will stall until a give writes to it?

Look at the #define for vSemaphoreCreateBinary in semphr.h. After it is created it is then given so a take will pass without blocking.