DOCS-687 new Concurrency FAQ

bin/builder_data.py

@@ -83,6 +83,7 @@
      ('$(rst-include)/table-sql-to-mongo-select-examples', 'sql'),
      ('$(rst-include)/table-sql-to-mongo-update-examples', 'sql'),
      ('$(rst-include)/table-sql-to-mongo-delete-examples', 'sql'),
+     ('$(rst-include)/table-lock-behavior-per-operation', 'concurrency'),
 ]
 
 sphinx = [

source/faq/concurrency.txt

@@ -0,0 +1,187 @@
+================
+FAQ: Concurrency
+================
+
+.. default-domain:: mongodb
+
+MongoDB allows concurrent operations by using locks to ensure that
+writes to a document occur without data loss. Locks ensure that all
+writes occur either in full or not at all.
+
+Locks also ensure that writes to a document or by a single connection
+occur in a predetermined order.
+
+.. contents:: Frequently Asked Questions:
+   :backlinks: none
+   :local:
+
+.. _faq-concurrency-locking:
+
+What type of locking does MongoDB use?
+--------------------------------------
+
+MongoDB uses a readers-writer lock that allows concurrent reads access
+to a database but gives exclusive access to a single write operation.
+
+When a read holds the lock, other reads can share it. However, when
+a write holds the lock, the write holds the lock exclusively. No other
+read nor write can share the lock.
+
+Locks are "writer greedy," which means writes have preference over
+reads. When both a read and write are waiting for a lock, MongoDB
+grants the lock to the write.
+
+.. versionchanged:: 2.2
+   Beginning with version 2.2, MongoDB implements locks on a
+   per-database basis. If you have six databases and one takes a write
+   lock, the other five are still available for reads. In versions prior
+   to 2.2, MongoDB implements locks per :program:`mongod` instance.
+
+For more information on locks, see the :data:`locks` documentation.
+
+How do I view lock status?
+--------------------------
+
+To view what operations are running and what each operation has locked,
+use :method:`currentOp() <db.currentOp()`>.
+
+To view lock status, use :method:`serverStatus() <db.serverStatus()>`.
+Optionally, you also can use the :doc:`mongotop </reference/mongotop>`
+utility, the :doc:`mongostat </reference/mongostat>` utility, or the
+`MongoDB Monitoring Service (MMS) <http://mms.10gen.com/>`_.
+
+To terminate an operation, use :method:`killOp() <db.killOp()>`.
+
+Does a read or write ever yield the lock?
+-----------------------------------------
+
+A read or write operation will yield a lock when MongoDB attempts to read data
+that is not yet in memory. When MongoDB attempts to read data not in memory,
+a :term:`page fault` occurs.
+
+.. versionchanged:: 2.2
+   MongoDB tracks the contents of memory and predicts whether data is
+   available before performing a read. If data is predicted to be
+   unavailable, a read operation yields its lock and requests that
+   MongoDB download the data to memory. Once data is available in
+   memory, the read retakes the lock and completes the operation.
+
+Which operations lock the database?
+-----------------------------------
+
+The following table lists common database operations and the types of
+locks they use.
+
+.. todo In the table below (in the include), the issue of blocked
+   JavaScript might no longer apply in version 2.4, which will use V8.
+
+.. include:: /includes/table-lock-behavior-per-operation.rst
+
+Which administrative commands lock the database?
+------------------------------------------------
+
+Certain administrative commands can exclusively lock the database for
+extended periods of time. On very large databases, consider taking the
+the :program:`mongod` instance offline so that clients are not affected.
+For example, if a :program:`mongod` is part of a :term:`replica set`,
+take the :program:`mongod` offline and let other members of the set
+service load while maintenance is in progress.
+
+The following administrative commands can exclusively lock the database for
+extended periods:
+
+- :method:`ensureIndex() <db.collection.ensureIndex()>`, when issued
+  *without* setting ``background`` to ``true``
+- :dbcommand:`reIndex`
+- :dbcommand:`compact`
+- :method:`repair.Database() <db.repairDatabase()>`
+- :method:`createCollection() <db.createCollection()>`, when creating a
+  very large (many gigabytes) capped collection
+- :method:`validate() <db.collection.validate()>`
+- :method:`db.copyDatabase()`. This command can lock all databases. See
+  :ref:`faq-concurrency-lock-multiple-dbs`.
+
+The :method:`group() <db.collection.group()>` command takes a read lock
+and does not allow any other threads to execute JavaScript while it is
+running.
+
+.. todo The above sentence might no longer apply in version 2.4, which will use V8.
+
+The following administrative commands lock the database but are fast and
+do not excessively block the system:
+
+- :method:`dropIndex() <db.collection.dropIndex()>`
+- :method:`getLastError() <db.collection.getLastError()>`
+- :method:`isMaster() <db.isMaster()>`
+- :method:`rs.status()` and :dbcommand:`replSetGetStatus`
+- :method:`serverStatus() <db.serverStatus()>`
+- :method:`auth() <db.auth()>`
+- :method:`addUser() <db.addUser()>`
+
+.. _faq-concurrency-lock-multiple-dbs:
+
+Does a MongoDB operation ever lock more than one database?
+----------------------------------------------------------
+
+The following MongoDB operations lock multiple databases:
+
+- :method:`db.copyDatabase()` performs a global lock.
+
+- Journaling, which is an internal operation, locks all databases for
+  short intervals. The journal is shared by all databases.
+
+- :ref:`User authentication <security-authentication>` locks the
+  ``admin`` database as well as the database the user is accessing.
+
+- Writes to a replica set's :term:`primary` lock both the database
+  receiving the writes and the ``local`` database, the latter in order
+  to record the writes to the primary's :term:`oplog`.
+
+How does sharding affect concurrency?
+-------------------------------------
+
+:term:`Sharding <sharding>` improves concurrency by distributing
+collections over multiple :program:`mongod` instances, allowing shard
+servers (i.e. :program:`mongos` processes) to perform any number of
+operations concurrently to the various downstream :program:`mongod`
+instances.
+
+Each :program:`mongod` instance is independent of the others in the
+shard cluster and uses the MongoDB :ref:`readers-writer lock
+<faq-concurrency-locking>`). The operations on one :program:`mongod`
+instance do not block the operations on any others.
+
+.. _faq-concurrency-replication:
+
+How does concurrency affect a replica set primary?
+--------------------------------------------------
+
+In :term:`replication`, when MongoDB writes to a collection on the
+:term:`primary`, MongoDB also writes to the primary's :term:`oplog`.
+Therefore, MongoDB must lock both the collection's database and the
+oplog's database. The oplog is located in the ``local`` database. Both
+must be locked at same time to keep the operation atomic. It is an
+all-or-nothing operation.
+
+How does concurrency affect secondaries?
+----------------------------------------
+
+In :term:`replication`, MongoDB does not apply writes serially to
+:term:`secondaries <secondary>`. Secondaries collect oplog entries in
+batches and then apply those batches in parallel. Secondaries do not
+allow reads while applying the batch writes.
+
+A primary might apply multiple writes to different databases
+simultaneously. When the primary finishes each write, it assigns each write
+to a place in the oplog, giving the writes a specific order.
+
+The secondaries apply the writes in the order in the oplog.
+
+MongoDB can apply several writes in parallel on replica set secondaries.
+This is done in two phases:
+
+- A prefetch phase occurs during a read lock. During this phase other
+  clients may execute queries.
+
+- A thread pool using write locks applies a batch of writes in a
+  coordinated write phase.

source/includes/table-lock-behavior-per-operation.yaml

@@ -0,0 +1,46 @@
+# table structure. all content symbolic.
+section: layout
+header: [ meta.header1, meta.header2 ]
+rows:
+  - 1: [ content.op1, content.lock1 ]
+  - 2: [ content.op2, content.lock2 ]
+  - 3: [ content.op3, content.lock3 ]
+  - 4: [ content.op4, content.lock4 ]
+  - 5: [ content.op5, content.lock5 ]
+  - 6: [ content.op6, content.lock6 ]
+  - 7: [ content.op7, content.lock7 ]
+  - 8: [ content.op8, content.lock8 ]
+  - 9: [ content.op9, content.lock9 ]
+---
+# table metadata, as meta.<key>
+section: meta
+header1: "Operation"
+header2: "Lock Type"
+---
+# table content, as content.<key>
+section: content
+op1: Issue a query
+lock1: Read lock
+op2: Get more data from a :term:`cursor`
+lock2: Read lock
+op3: Insert data
+lock3: Write lock
+op4: Remove data
+lock4: Write lock
+op5: Update data
+lock5: Write lock
+op6: :term:`Map-reduce <map-reduce>`
+lock6: |
+       Read lock and write lock, unless operations are specified as
+       non-atomic. Portions of map-reduce jobs can run concurrently.
+op7: Create an index
+lock7: |
+       Building an index in the foreground, which is the default, locks
+       the database for extended periods of time.
+op8: :method:`db.eval()`
+lock8: |
+       Write lock or no lock. If this is used without the ``nolock``
+       option, all JavaScript is blocked.
+op9: :method:`aggregate() <db.collection.aggregate()>`
+lock9: Read lock
+...