With the upcoming possibilities of CSOM for SharePoint 2013, setting de default value Managed Metadata Columns is often part of the provisioning of site collections, apps or documents.

The code snippet below shows how the default value can be set assuming that you already have retrieved the Field definition of the site column wherefore you want to set the default value (and that you know the term label and ID).

You need a reference to the Microsoft.SharePoint.Client, Microsoft.SharePoint.Client.Runtime and Microsoft.SharePoint.Client.Taxonomy for this code to work:

string termLabel = "My Term";
Guid termId = new Guid("{590861F6-2E60-4198-A9CC-7D39158BF66E}");

TaxonomyField taxonomyField = Context.CastTo<TaxonomyField>(field);
Context.Load(taxonomyField, t => t.DefaultValue);
Context.ExecuteQuery(); // Get the Taxonomy Field

TaxonomyFieldValue defaultValue = new TaxonomyFieldValue();
defaultValue.WssId = -1;
defaultValue.Label = termLabel;
// GUID should be stored lowercase, otherwise it will not work in Office 2010
defaultValue.TermGuid = termId.ToString().ToLower();

// Get the Validated String for the taxonomy value
var validatedValue = taxonomyField.GetValidatedString(defaultValue);
Context.ExecuteQuery(); 

// Set the selected default value for the site column
taxonomyField.DefaultValue = validatedValue.Value;
taxonomyField.UserCreated = false;
taxonomyField.UpdateAndPushChanges(true);
Context.ExecuteQuery();