Archive mail somewhere else
Stream messages into your own warehouse or object storage for retention and analysis.
Why anyone does this
Retention rules, legal hold and analytics all want mail somewhere you control. The mailbox is the wrong place to keep years of it, and the wrong place to query it from.
What a key needs
Grant these and nothing else. A key can never do more than the person who created it, and one that carries only what the job needs is one you can leave running.
The flow
-
List the folders once
Folder ids are what everything else is keyed on, and they do not change. Read them at the start of a run rather than per message.
GET
/v1/folders -
Page through each folder
Newest first, with a cursor. Keep the cursor, not the page number: messages arriving mid-run shift an offset and would make you skip one.
GET
/v1/messages -
Fetch the ones you have not seen
The summary carries no recipients and no body. Fetch the message for those, and only for ids your store does not already hold.
GET
/v1/messages/:id -
Pull attachments separately
Attachment bytes are their own call, so archiving a mailbox does not drag every large file through the message responses.
GET
/v1/messages/:id/attachments/:part
The shape of it
Short enough to read in one go, and close enough to run that the gaps are obvious.
# Every folder, then every message in it.
folders=$(curl -s https://api.veruapis.com/v1/folders \
-H "Authorization: Bearer $KEY" | jq -r '.data[].id')
for folder in $folders; do
cursor=""
while :; do
page=$(curl -s "https://api.veruapis.com/v1/messages?folder_id=$folder&limit=200&cursor=$cursor" \
-H "Authorization: Bearer $KEY")
echo "$page" | jq -c '.data[]' >> archive.ndjson
cursor=$(echo "$page" | jq -r '.meta.next_cursor // empty')
[ -z "$cursor" ] && break
done
done
What catches people out
Every one of these came from somebody getting it wrong.
- Store the message id, not its position. A folder is not stable enough to page by offset.
- A message moved between folders keeps its id, so deduplicate on the id or you will archive it twice.
- Run it incrementally. A full sweep of a busy mailbox will meet the daily quota long before it meets the end.