# 4.2. Email Database

The python bindings to libnotmuch define `notmuch.Thread` and `notmuch.Message`, which unfortunately are very fragile. Alot defines the wrapper classes `alot.db.Thread` and `alot.db.Message` that use an `manager.DBManager` instance to transparently provide persistent objects.

`alot.db.Message` moreover contains convenience methods to extract information about the message like reformated header values, a summary, decoded and interpreted body text and a list of `Attachments`.

The central `UI` instance carries around a `DBManager` object that is used for any lookups or modifications of the email base. `DBManager` can directly look up `Thread` and `Message` objects and is able to postpone/cache/retry writing operations in case the Xapian index is locked by another process.

## 4.2.1. Database Manager

class alot.db.manager.DBManager(path=None, ro=False, config=None)

  Keeps track of your index parameters, maintains a write-queue and lets you look up threads and messages directly to the persistent wrapper classes.

  Parameters:

    - path (str) – absolute path to the notmuch index

    - ro (bool) – open the index in read-only mode

    - config – absolute path to the notmuch config file

  add_message(path, tags=None, afterwards=None)

    Adds a file to the notmuch index.

    Parameters:

      - path (str) – path to the file

      - tags (list of str) – tagstrings to add

      - afterwards (callable or None) – callback to trigger after adding

  collect_tags(querystring)

    returns tags of messages that match querystring

  count_messages(querystring)

    returns number of messages that match querystring

  count_threads(querystring)

    returns number of threads that match querystring

  flush()

    write out all queued write-commands in order, each one in a separate `atomic` transaction.

    If this fails the current action is rolled back, stays in the write queue and an exception is raised. You are responsible to retry flushing at a later time if you want to ensure that the cached changes are applied to the database.

    Exception:

      `DatabaseROError` if db is opened read-only

    Exception:

      `DatabaseLockedError` if db is locked

  get_all_tags()

    returns all tagsstrings used in the database :rtype: list of str

  get_message(mid)

    returns `Message` with given message id (str)

  get_named_queries()

    returns the named queries stored in the database. :rtype: dict (str -> str) mapping alias to full query string

  get_thread(tid)

    returns `Thread` with given thread id (str)

  get_threads(querystring, sort='newest_first')

    asynchronously look up thread ids matching querystring.

    Parameters:

      - querystring (str.) – The query string to use for the lookup

      - sort – Sort order. one of [‘oldest_first’, ‘newest_first’, ‘message_id’, ‘unsorted’]

    Returns:

      a pipe together with the process that asynchronously writes to it.

    Return type:

      (`multiprocessing.Pipe`, `multiprocessing.Process`)

  remove_message(message, afterwards=None)

    Remove a message from the notmuch index

    Parameters:

      - message (`Message`) – message to remove

      - afterwards (callable or None) – callback to trigger after removing

  remove_named_query(alias, afterwards=None)

    remove a named query from the notmuch database.

    Parameters:

      - alias (str) – name of shortcut

      - afterwards (callable or None) – callback to trigger after adding the alias

  save_named_query(alias, querystring, afterwards=None)

    add an alias for a query string.

    These are stored in the notmuch database and can be used as part of more complex queries using the syntax “query:alias”. See notmuch-search-terms(7) for more info.

    Parameters:

      - alias (str) – name of shortcut

      - querystring (str) – value, i.e., the full query string

      - afterwards (callable or None) – callback to trigger after adding the alias

  tag(querystring, tags, afterwards=None, remove_rest=False)

    add tags to messages matching querystring. This appends a tag operation to the write queue and raises `DatabaseROError` if in read only mode.

    Parameters:

      - querystring (str) – notmuch search string

      - tags (list of str) – a list of tags to be added

      - afterwards (callable) – callback that gets called after successful application of this tagging operation

      - remove_rest (bool) – remove tags from matching messages before tagging

    Exception:

      `DatabaseROError`

    Note

    This only adds the requested operation to the write queue. You need to call `DBManager.flush()` to actually write out.

  toggle_tags(querystring, tags, afterwards=None)

    toggles tags from messages that match querystring. This appends a toggle operation to the write queue and raises `DatabaseROError` if in read only mode.

    Parameters:

      - querystring (str) – notmuch search string

      - tags (list of str) – a list of tags to be added

      - afterwards (callable) – callback that gets called after successful application of this tagging operation

    Exception:

      `DatabaseROError`

    Note

    This only adds the requested operation to the write queue. You need to call `DBManager.flush()` to actually write out.

  untag(querystring, tags, afterwards=None)

    removes tags from messages that match querystring. This appends an untag operation to the write queue and raises `DatabaseROError` if in read only mode.

    Parameters:

      - querystring (str) – notmuch search string

      - tags (list of str) – a list of tags to be added

      - afterwards (callable) – callback that gets called after successful application of this tagging operation

    Exception:

      `DatabaseROError`

    Note

    This only adds the requested operation to the write queue. You need to call `DBManager.flush()` to actually write out.

## 4.2.2. Errors

class alot.db.errors.DatabaseError

class alot.db.errors.DatabaseROError

  cannot write to read-only database

class alot.db.errors.DatabaseLockedError

  cannot write to locked index

class alot.db.errors.NonexistantObjectError

  requested thread or message does not exist in the index

## 4.2.3. Wrapper

class alot.db.Thread(dbman, thread)

  A wrapper around a notmuch mailthread (`notmuch2.Thread`) that ensures persistence of the thread: It can be safely read multiple times, its manipulation is done via a `alot.db.DBManager` and it can directly provide contained messages as `Message`.

  Parameters:

    - dbman (`DBManager`) – db manager that is used for further lookups

    - thread (`notmuch2.Thread`) – the wrapped thread

  add_tags(tags, afterwards=None, remove_rest=False)

    add tags to all messages in this thread

    Note

    This only adds the requested operation to this objects `DBManager's` write queue. You need to call `DBManager.flush` to actually write out.

    Parameters:

      - tags (list of str) – a list of tags to be added

      - afterwards (callable) – callback that gets called after successful application of this tagging operation

      - remove_rest (bool) – remove all other tags

  get_authors()

    returns a list of authors (name, addr) of the messages. The authors are ordered by msg date and unique (by name/addr).

    Return type:

      list of (str, str)

  get_authors_string(own_accts=None, replace_own=None)

    returns a string of comma-separated authors Depending on settings, it will substitute “me” for author name if address is user’s own.

    Parameters:

      - own_accts (list of `Account`) – list of own accounts to replace

      - replace_own (bool) – whether or not to actually do replacement

    Return type:

      str

  get_messages()

    returns all messages in this thread as dict mapping all contained messages to their direct responses.

    Return type:

      dict mapping `Message` to a list of `Message`.

  get_newest_date()

    returns date header of newest message in this thread as `datetime`

  get_oldest_date()

    returns date header of oldest message in this thread as `datetime`

  get_replies_to(msg)

    returns all replies to the given message contained in this thread.

    Parameters:

      msg (`Message`) – parent message to look up

    Returns:

      list of `Message` or None

  get_subject()

    returns subject string

  get_tags(intersection=False)

    returns tagsstrings attached to this thread

    Parameters:

      intersection (bool) – return tags present in all contained messages instead of in at least one (union)

    Return type:

      set of str

  get_thread_id()

    returns id of this thread

  get_toplevel_messages()

    returns all toplevel messages contained in this thread. This are all the messages without a parent message (identified by ‘in-reply-to’ or ‘references’ header.

    Return type:

      list of `Message`

  get_total_messages()

    returns number of contained messages

  matches(query)

    Check if this thread matches the given notmuch query.

    Parameters:

      query (string) – The query to check against

    Returns:

      True if this thread matches the given query, False otherwise

    Return type:

      bool

  refresh(thread=None)

    refresh thread metadata from the index

  remove_tags(tags, afterwards=None)

    remove tags (list of str) from all messages in this thread

    Note

    This only adds the requested operation to this objects `DBManager's` write queue. You need to call `DBManager.flush` to actually write out.

    Parameters:

      - tags (list of str) – a list of tags to be added

      - afterwards (callable) – callback that gets called after successful application of this tagging operation

class alot.db.Message(dbman, msg, thread=None)

  a persistent notmuch message object. It it uses a `DBManager` for cached manipulation and lazy lookups.

  Parameters:

    - dbman (alot.db.DBManager) – db manager that is used for further lookups

    - msg (notmuch2.Message) – the wrapped message

    - thread (`Thread` or None) – this messages thread (will be looked up later if None)

  add_tags(tags, afterwards=None, remove_rest=False)

    adds tags to message

    Note

    This only adds the requested operation to this objects `DBManager's` write queue. You need to call `flush()` to write out.

    Parameters:

      - tags (list of str) – a list of tags to be added

      - afterwards (callable) – callback that gets called after successful application of this tagging operation

      - remove_rest (bool) – remove all other tags

  get_attachments()

    returns messages attachments

    Derived from the leaves of the email mime tree that and are not part of RFC 2015 syntax for encrypted/signed mails and either have Content-Disposition attachment or have Content-Disposition inline but specify a filename (as parameter to Content-Disposition).

    Return type:

      list of `Attachment`

  get_author()

    returns realname and address of this messages author

    Return type:

      (str,str)

  get_body_text()

    returns bodystring extracted from this mail

  get_date()

    returns Date header value as `datetime`

  get_datestring()

    returns reformated datestring for this message.

    It uses `SettingsManager.represent_datetime()` to represent this messages Date header

    Return type:

      str

  get_email()

    returns `email.email.EmailMessage` for this message

  get_filename()

    returns absolute path of message files location

  get_message_id()

    returns messages id (str)

  get_message_parts()

    yield all body parts of this message

  get_replies()

    returns replies to this message as list of `Message`

  get_tags()

    returns tags attached to this message as list of strings

  get_thread()

    returns the `Thread` this msg belongs to

  get_thread_id()

    returns id (str) of the thread this message belongs to

  has_replies()

    returns true if this message has at least one reply

  matches(querystring)

    tests if this messages is in the resultset for querystring

  remove_tags(tags, afterwards=None)

    remove tags from message

    Note

    This only adds the requested operation to this objects `DBManager's` write queue. You need to call `flush()` to actually out.

    Parameters:

      - tags (list of str) – a list of tags to be added

      - afterwards (callable) – callback that gets called after successful application of this tagging operation

## 4.2.4. Other Structures

class alot.db.attachment.Attachment(emailpart)

  represents a mail attachment

  Parameters:

    emailpart (`email.message.Message`) – a non-multipart email that is the attachment

  get_content_type()

    mime type of the attachment part

  get_data()

    return data blob from wrapped file

  get_filename()

    return name of attached file. If the content-disposition header contains no file name, this returns None

  get_mime_representation()

    returns mime part that constitutes this attachment

  get_size()

    returns attachments size in bytes

  save(path)

    save the attachment to disk. Uses `get_filename()` in case path is a directory

  write(fhandle)

    writes content to a given filehandle

class alot.db.envelope.Envelope(template=None, bodytext=None, headers=None, attachments=None, sign=False, sign_key=None, encrypt=False, tags=None, replied=None, passed=None, account=None)

  a message that is not yet sent and still editable.

  It holds references to unencoded! body text and mail headers among other things. Envelope implements the python container API for easy access of header values. So e[‘To’], e[‘To’] = ‘foo@bar.baz’ and ‘e.get_all(‘To’)’ would work for an envelope e..

  Parameters:

    - template (str) – if not None, the envelope will be initialised by `parsing` this string before setting any other values given to this constructor.

    - bodytext (str) – text used as body part

    - headers (dict (str -> [unicode])) – unencoded header values

    - attachments (list of `Attachment`) – file attachments to include

    - tags (list of str) – tags to add after successful sendout and saving this msg

    - replied (`Message`) – message being replied to

    - passed – message being passed on

    - account (`Account`) – account to send from

  add(key, value)

    add header value

  attach(attachment, filename=None, ctype=None)

    attach a file

    Parameters:

      - attachment (`Attachment` or str) – File to attach, given as `Attachment` object or path to a file.

      - filename – filename to use in content-disposition. Will be ignored if path matches multiple files

      - ctype (str) – force content-type to be used for this attachment

  construct_mail()

    Compiles the information contained in this envelope into a `email.Message`.

  get(key, fallback=None)

    secure getter for header values that allows specifying a fallback return string (defaults to None). This returns the first matching value and doesn’t raise KeyErrors

  get_all(key, fallback=None)

    returns all header values for given key

  parse_template(raw, reset=False, only_body=False, target_body='plaintext')

    Parse a template or user edited string to fills this envelope.

    Parameters:

      - raw (str) – the string to parse.

      - reset (str) – remove previous envelope content

      - only_body (bool) – do not parse headers

      - target_body – body text alternative this should be stored in; can be ‘plaintext’ or ‘html’

  account = None

    account to send from

  attachments = None

    list of `Attachments`

  body_html = None

    mail body (html) as unicode string

  body_txt = None

    mail body (plaintext) as unicode string

  headers = None

    dict containing the mail headers (a list of strings for each header key)

  tags = []

    tags to add after successful sendout

  tmpfile = None

    template text for initial content

## 4.2.5. Utilities

alot.db.utils.add_signature_headers(mail, sigs, error_msg)

  Add pseudo headers to the mail indicating whether the signature verification was successful.

  Parameters:

    - mail – `email.message.Message` the message to entitle

    - sigs – list of `gpg.results.Signature`

    - error_msg (`str` or None) – An error message if there is one, or None

alot.db.utils.clear_my_address(my_account, value)

  return recipient header without the addresses in my_account

  Parameters:

    - my_account (`Account`) – my account

    - value (list(str)) – a list of recipient or sender strings (with or without real names as taken from email headers)

  Returns:

    a new, potentially shortend list

  Return type:

    list(str)

alot.db.utils.decode_header(header, normalize=False)

  decode a header value to a unicode string

  values are usually a mixture of different substrings encoded in quoted printable using different encodings. This turns it into a single unicode string

  Parameters:

    - header (str) – the header value

    - normalize (bool) – replace trailing spaces after newlines

  Return type:

    str

alot.db.utils.decrypted_message_from_bytes(bytestring, session_keys=None)

  Create a Message from bytes.

  Parameters:

    - bytestring (bytes) – an email message as raw bytes

    - session_keys – a list OpenPGP session keys

alot.db.utils.ensure_unique_address(recipients)

  clean up a list of name,address pairs so that no address appears multiple times.

alot.db.utils.extract_body_part(body_part)

  Returns a string view of a Message.

alot.db.utils.extract_headers(mail, headers=None)

  returns subset of this messages headers as human-readable format: all header values are decoded, the resulting string has one line “KEY: VALUE” for each requested header present in the mail.

  Parameters:

    - mail (`email.message.EmailMessage`) – the mail to use

    - headers (list of str) – headers to extract

alot.db.utils.formataddr(pair)

  this is the inverse of email.utils.parseaddr: other than email.utils.formataddr, this - will not re-encode unicode strings, and - will re-introduce quotes around real names containing commas

alot.db.utils.get_body_part(mail, mimetype=None)

  Returns an EmailMessage.

  This consults prefer_plaintext to determine if a “text/plain” alternative is preferred over a “text/html” part.

  Parameters:

    mail (`email.message.EmailMessage`) – the mail to use

  Returns:

    The combined text of any parts to be used

  Return type:

    str

alot.db.utils.get_params(mail, failobj=None, header='content-type', unquote=True)

  Get Content-Type parameters as dict.

  RFC 2045 specifies that parameter names are case-insensitive, so we normalize them here.

  Parameters:

    - mail – `email.message.Message`

    - failobj – object to return if no such header is found

    - header – the header to search for parameters, default

    - unquote – unquote the values

  Returns:

    a dict containing the parameters

alot.db.utils.remove_cte(part, as_string=False)

  Interpret MIME-part according to it’s Content-Transfer-Encodings.

  This returns the payload of part as string or bytestring for display, or to be passed to an external program. In the raw file the payload may be encoded, e.g. in base64, quoted-printable, 7bit, or 8bit. This method will look for one of the above Content-Transfer-Encoding header and interpret the payload accordingly.

  Incorrect header values (common in spam messages) will be interpreted as lenient as possible and will result in INFO-level debug messages.

  ..Note:: All this may be depricated in favour of

    email.contentmanager.raw_data_manager (v3.6+)

  Parameters:

    - part (email.message.EmailMessage) – The part to decode

    - as_string (bool) – If true return a str, otherwise return bytes

  Returns:

    The mail with any Content-Transfer-Encoding removed

  Return type:

    Union[str, bytes]

alot.db.utils.render_part(part, field_key='copiousoutput')

  renders a non-multipart email part into displayable plaintext by piping its payload through an external script. The handler itself is determined by the mailcap entry for this part’s ctype.