It is often necessary in current projects that your mailbox is as empty as possible and people who ask to unsubscribe are helped. Normal method of unsubscribing is just via a form on your website, but usually people work in different ways. They throw message in spam folder and then loopback notification comes to you via Mail provider via Abuse report or via log file at SendGrid or MailChimp. Or people simply forward or reply to a message with a question to delete all data, that can become a very method of managing a mailbox where many automatic notifications are sent. We'll cover standard ways we use to keep feedback loop cleaner in this post.
This document is based on internal documentation and has been made a bit generic so that it can be used in various other cases. Original document can be found in documents cloud with name:
Unsubscribe cyclus from Sendgrid, MailChimp Spam and bounce
Checklijst
Default unsubscribe Form
Mailbox map
Blacklisted global
Bounce via sending providers
Spam reports via loopback sending providers
First method Exporting Gmail emails
First source of requests that is the mailbox itself where people forward and ask for their data to be deleted. Sometimes they don't want to click on a link or search or expect a complicated process but that's the fact quite a few unsubscribes end up in mailbox and in order not to manually process such requests one can automate Gmail in various ways.
Export data per label and then read this local mbox file read all messages and unsubscribe senders.
Exporting data might work even better than giving access to your account with script. Because then we can edit archive even better locally.
You can manage all your exports from this page. Create find where they are now etc..
https://takeout.google.com/
This method is good if you want to initiate it manually and it is more secure.
The other real time downloading of messages is not so good I think because you have to make your account less secure, there is no possibility to allow access only by IP, for example. When you open IMAP it is open to everyone and all that is left is your password.
Export is done almost immediately without delay as first mentioned hours or days.. That took 1 minute and I already had my files.. Imported emails via Drive with 1 label..
Now I have a zipped mbox file that I should now be able to read and write out all messages in this file so that they don't exist anymore.
With this library it seems to be possible so we will test first and see how effective it is. For now this solution I think is the best option.. So how do I go about that just add all people in global unsubscribe list so they can leave all other services or work by list. People who have retired or complain can be completely removed from the system, others who have found work can still stay in system but put on inactive so that we ask back in 1 year if they are available.
https://docs.python.org/3/library/mailbox.html
Code will look something like this depending on what exactly is needed.
An example of script that can be used and further extended to a Django command or a similar script with a correct functional purpose.
# -*- coding: utf-8 -*-
#!/usr/bin/python
#
# Copyright (C) 2021 Web App Development
#
"""
"""
__author__ = 'sergej[A]dergatsjev[D]be (Sergej Dergatsjev)'
from django.db import models, transaction, IntegrityError
from django.core.management.base import BaseCommand
from django.template.defaultfilters import slugify
from crawler import utils
import re
from time import sleep
import sys, traceback
from newsletter.models import Subscriber
import mailbox
class Command(BaseCommand):
def handle(self, *args, **options):
"""
Entery point
"""
#import pdb;pdb.set_trace()
for message in mailbox.mbox("data/Unsubscribe.mbox"):
subject = message['subject']
from_email = message['From']
print("-----------------")
print(from_email)
if from_email.find('MAILER-DAEMON@secure-mail.be') != -1:
from_email = self.parse_from_email(message.as_string(), "for <([\w\-][\w\-\.]+@[\w\-][\w\-\.]+[a-zA-Z]{1,4})>")
print(from_email)
self.unsubscribe(from_email, "Undelivered")
elif from_email.find('<') != 1:
from_email = self.parse_from_email(from_email, "<([\w\-][\w\-\.]+@[\w\-][\w\-\.]+[a-zA-Z]{1,4})>")
self.unsubscribe(from_email, "Unsubscribe Mailbox")
else:
self.unsubscribe(from_email, "Unsubscribe Mailbox")
print(subject)
#email = utils.normalization_email(cols[0])
def parse_from_email(self, text, pattern):
pattern = re.compile(pattern)
result = re.findall(pattern, text)
for item in result:
if "vindazo.be" in item:
result.remove(item)
if len(result) > 0:
return result[0]
else:
return None
def unsubscribe(self, from_email, status):
if from_email:
subscriber = None
print(from_email)
try:
subscriber = Subscriber.objects.get(email=from_email, has_unsubscribe=False)
except:
try:
subscriber = Subscriber.objects.get(email=from_email)
print("All ready unsubscribed")
return None
except:
print("Email not found")
if subscriber:
subscriber.has_unsubscribe = True
subscriber.status = status
subscriber.save()
return from_email
Second example.. More complex case with different regular expressions to find "from mail"
# -*- coding: utf-8 -*-
#!/usr/bin/python
#
# Copyright (C) 2020 Web App Development EOOD
#
"""
Unsubscribe via profile
"""
import imaplib
import re
import requests
from datetime import datetime, date
import sys, traceback, os
from time import sleep
from spontaneousmail import utils
from django.core.management.base import BaseCommand
from spontaneousmail.models import SpontaneousProfile
import mailbox
class Command(BaseCommand):
verbose = True
timeout = 5
count = 0
mailbox = "data/Unsubscribe.mbox"
def handle(self, *args, **options):
"""
Unsubscribe via mailbox
"""
contacts = self.read_contacts()
count = 1
for contact in contacts:
print(count)
print(contact['email'])
count = count + 1
utils.unsubscribe(contact['email'], "Unsubscribe_mailbox")
def parse_from_email(self, text, pattern):
pattern = re.compile(pattern)
result = re.findall(pattern, text)
for item in result:
if ("vacaturemails.com" in item) or ("vacatures.today" in item):
result.remove(item)
if len(result) > 0:
return result[0]
else:
return None
def read_contacts(self):
"""
"""
#import pdb;pdb.set_trace()
contacts = []
count = 0
for message in mailbox.mbox(self.mailbox):
count = count + 1
subject = message['subject']
from_email = message['From']
print(count)
print(from_email)
if from_email.find('mailer-daemon') != -1:
# for <info@vacaturemails.com>
#import pdb;pdb.set_trace()
from_email = self.parse_from_email(message.as_string(), "for <([\w\-][\w\-\.]+@[\w\-][\w\-\.]+[a-zA-Z]{1,4})>")
if not from_email:
# Final-Recipient: rfc822; dd.ff@europe.com
from_email = self.parse_from_email(message.as_string(), "Final-Recipient: rfc822; ([\w\-][\w\-\.]+@[\w\-][\w\-\.]+[a-zA-Z]{1,4})")
elif from_email.find('<') != -1:
from_email = self.parse_from_email(from_email, "<([\w\-][\w\-\.]+@[\w\-][\w\-\.]+[a-zA-Z]{1,4})>")
if not from_email:
#import pdb;pdb.set_trace()
from_email = self.parse_from_email(message.as_string(), "To:.*([\w\-][\w\-\.]+@[\w\-][\w\-\.]+[a-zA-Z]{1,4})")
#import pdb;pdb.set_trace()
#print(subject)
if from_email:
contacts.append({"email":from_email})
else:
#import pdb;pdb.set_trace()
f = open("/tmp/emails/" + str(count), "w")
try:
f.write(message.as_string())
except:
traceback.print_exc(file=sys.stdout)
f.close()
return contacts
Unsubscribe Real Time via IMAP Gmail
Then the next method is about connection via IMAP and direct access to the correct label and processing all messages under a correct label such as Unsubscribe for example. People who manage mailbox move all mails under this label and script comes once a day to check if there is something to process and move in folder with all other ready messages in archive.
I don't recommend this way because it exposes your account to atteres from outside. But you can do it if you want. We've done it once for less important accounts too.
You will have an issues with login via IMAP.
b'[AUTHENTICATIONFAILED] Invalid credentials (Failure)'
python3 manage.py unsubscribe_imap
https://accounts.google.com/b/0/DisplayUnlockCaptcha
Occasionally Gmail will stop working and you can see a Postie error message that says something like “[ALERT] Please log in via your web browser”. This means you must log into the account Postie is trying to access via https://gmail.com where you will be asked to confirm a login from wherever your server is located.
If this doesn’t work try http://www.google.com/accounts/DisplayUnlockCaptcha
Turn off 2-Step Verification https://support.google.com/accounts/answer/1064203?hl=en If you are still having difficulty you may need to change the Google “Less secure apps” setting. Be sure you are only signed into the gmail account that postie will use then go to https://www.google.com/settings/security/lesssecureapps and make sure it is set to “Turn off”
While the wording on this page makes it sound dangerous, what it really means is “allow traditional authentication” which uses very high levels of encryption.
Finally check these pages for other options:
https://support.google.com/accounts/answer/6009563
https://support.google.com/mail/answer/14257
Code voorbeeld:
def login():
im = imaplib.IMAP4_SSL(INBOX_IMAP, 993)
im.login(INBOX_USER, INBOX_PASS)
return im
def unsubscribe(mailbox, im):
typ, data = im.search(None, 'ALL')
for num in data[0].split():
typ, msg = im.fetch(num, '(RFC822)')
unsubscribe_message(msg[0][1].decode("utf-8"), num, mailbox)
im = login()
unsubscribe("Unsubscribe", im)
# -*- coding: utf-8 -*-
#!/usr/bin/python
#
# Copyright (C) 2020 Web App Development EOOD
# Online Solutions Group BVBA
"""
Unsubscribe via profile
"""
import imaplib
import re
import requests
from datetime import datetime, date
import sys, traceback, os
from time import sleep
from spontaneousmail import utils
from django.core.management.base import BaseCommand
from spontaneousmail.models import SpontaneousProfile
INBOX_IMAP = 'imap.gmail.com'
INBOX_USER = ''
INBOX_PASS = ''
class Command(BaseCommand):
def handle(self, *args, **options):
"""
Unsubscribe via imap
"""
im = login()
unsubscribe("Unsubscribe", im)
def login():
im = imaplib.IMAP4_SSL(INBOX_IMAP, 993)
im.login(INBOX_USER, INBOX_PASS)
return im
def is_own(item):
OWN = ["vacaturemails.com", "vacatures.today"]
for own_email in OWN:
if item.find(own_email) != -1:
return True
return False
def get_to(message):
#import pdb;pdb.set_trace()
pattern = re.compile("From: [^\r]*\r\nTo: ([^\r]*)\r\nDate: ")
result = re.findall(pattern, message)
for item in result:
if not is_own(item):
return item
pattern = re.compile("\r\nTo:([^\r]*)\r\n")
result = re.findall(pattern, message)
for item in result:
if not is_own(item):
return item
def get_from(message):
pattern = re.compile("From: ([^\r]*)\r\n")
result = re.findall(pattern, message)
for item in result:
if not is_own(item):
return item
def clean(email):
if email.find("<") != -1:
pattern = re.compile("<([^>]*)>")
return re.findall(pattern, email)[0]
return email.strip()
def unsubscribe_remove(mailbox, im):
# Use onther map to move emails first
# Or if it realy should be deleted
print(im.select(mailbox))
typ, data = im.search(None, 'ALL')
for num in data[0].split():
typ, msg = im.fetch(num, '(RFC822)')
unsubscribe_message(msg[0][1], num, mailbox)
im.store(num, '+FLAGS', '\\Deleted')
im.expunge()
def remove_emails(mailbox, im):
# Like priviews but not used in this script only for exampole saved
print(im.select(mailbox))
typ, data = im.search(None, 'ALL')
import pdb;pdb.set_trace()
count = 0
max_count = 100
total = get_count(im)
while total > 0:
total = get_count(im)
count = count + 1
print(str(count))
if count == max_count:
im.expunge()
count = 1
try:
im.store(count, '+FLAGS', '\\Deleted')
except:
traceback.print_exc(file=sys.stdout)
im.expunge()
im.expunge()
def unsubscribe(mailbox, im):
# Usage like
#for mailbox in FOR_UNSUBSCRIBE:
# unsubscribe(mailbox, im)
print(im.select(mailbox))
import pdb;pdb.set_trace()
typ, data = im.search(None, 'ALL')
for num in data[0].split():
typ, msg = im.fetch(num, '(RFC822)')
unsubscribe_message(msg[0][1].decode("utf-8"), num, mailbox)
#im.expunge() deleted items from selected mailbox
def unsubscribe_message(message, num, label):
email = get_to(message)
if email == None:
email = get_from(message)
if email:
email = clean(email)
print(email)
# Here unsubscribe logic from database or via API url
#print(response)
utils.unsubscribe(email, "Unsubscribe_imap")
def get_count(im):
try:
typ, data = im.search(None, 'ALL')
except:
traceback.print_exc(file=sys.stdout)
im = login()
typ, data = im.search(None, 'ALL')
return len(data[0].split())
SendGrid MailChip unsubscribe automation
SendGrid and MailChip have the option to export their logs so you can export blocked and/or bounced messages from these services and unsubscribe automatically. They use CSV which is text format divided with koma.
Note they do not have the same format and are not in the same order so check the export order carefully.
Code example:
def handle(self, *args, **options):
"""
"""
files = [
'data/bounce/suppression_bounces.csv'
]
for file in files:
print(file)
f = open(file, "r")
items = f.readlines()
self.unsubsribe_emails(items)
def unsubsribe_emails(self, items):
import pdb;pdb.set_trace()
for item in items:
time.sleep(0.1)
item = item.split(',')[2]
try:
term = item.strip().lower()
print(term)
utils.unsubscribe(term, "Bounce")
except:
traceback.print_exc(file=sys.stdout)
More examples see in intern commands of mailing application in CRM.
For example:
mailing/management/commands/importsendgridspam.py
mailing/management/commands/unsubscribe_suppression.py
mailing/management/commands/importsendgridbounce.py
mailing/management/commands/collect_suppression.py
mailing/management/commands/copyemailsfromstatus.py
If you start to unsubscribe automatically then look very carefully at status and message because some messages mean nothing but blocked and that is per domain or even message level and bounce mailbox is still OK but sender is not OK and he is blocked and this bounce actually belongs in category blocked en not bounce.. Bounce is when mailbox is closed or overquite. So look good, such message from from skynet.be provider is about your blacklisted IP, Domain or message by skynet provider..
550 #5.1.0 Address rejected.
When you filter this messages and unsubscribe it from list, unsubscribe it not on global level but only per domain and message.
To read messages in mail log from big logfile about 50 GB.. For example, if you have a configuration that contains all baskets or even years, it could be that it is not readable with VIM. Then you can cut and read the last 100K from this file.
du -h /var/log/mail.log
50G /var/log/mail.log
tail -n 100000 /var/log/mail.log > /tmp/log
Unsubscribe via Firebase form
We have several ways to accomplish this task. We can use the REST API and lib which was made 5 years ago.
https://firebase.google.com/docs/database/rest/start
https://github.com/thisbejim/Pyrebase
Or even older like
https://github.com/ozgur/python-firebase
So it's not recommended that we write new applications and use Realtime Database for my purposes, I've found Firestore Database works much better, and I'm trying to limit or eliminate this old use altogether in the future.
Next option is also an export of data. If for example you only need to perform this unsubscribe operation 1 time per month then you can export data and unsubscribe.
But we will automate immediately in a normal way because I think this application will be used for a long time and proper automation is needed from the start.
Pyrebase was written for python 3 and will not work correctly with python 2.
The first step is the configuration.
import pyrebase
config = {
"apiKey": "AIzaSyAfGNahILALZrp25B_-l97-dfdfdf",
"authDomain": "vacatures.firebaseapp.com",
"databaseURL": "https://vacatures.firebaseio.com",
"storageBucket": "vacatures.appspot.com"
"serviceAccount": "path/to/serviceAccountCredentials.json"
}
firebase = pyrebase.initialize_app(config)
In other words you can find all config information in Web or App configs.
https://firebase.google.com/docs/projects/api-keys
var firebaseConfig = {
apiKey: "AIzaSyAfGNahILALZsdfsdfllSdd-dfdf",
authDomain: "vacature.firebaseapp.com",
databaseURL: "https://vacature.firebaseio.com",
projectId: "vacature",
storageBucket: "vacatures.appspot.com",
messagingSenderId: "4455454",
appId: "1:479857347200:web:343434",
measurementId: "G-DFDF4545"
};
But if we need full Admin access we can still
"serviceAccount": "path/to/serviceAccountCredentials.json"
In this way we can read data that is restricted with rights for other users.
With this wrapper you can address a number of services for example auth service.
https://github.com/thisbejim/Pyrebase#authentication
For example, it is possible to create or delete a user automatically.
So if your unsubscribe logic includes removing users then you can do it automatically via this wrapper reference.
At the moment we will only remove people who have unsubscribed from mailing and leave users unchanged.
Example
class Command(BaseCommand):
"""
Unsubscriebe emails from firebase database
"""
def handle(self, *args, **options):
"""
"""
firebase = pyrebase.initialize_app(config)
db = firebase.database()
contacts = db.child("unsubscribe").get()
#import pdb;pdb.set_trace()
for contact in contacts.each():
self.unsubsribe_emails(contact.val())
def unsubsribe_emails(self, item):
#import pdb;pdb.set_trace()
try:
term = item['email'].strip().lower()
print(term)
utils.unsubscribe(term, "Unsubscribe form")
except:
traceback.print_exc(file=sys.stdout)
This document is based on internal documentation and has been made a bit generic so that it can be used in various other cases. Original document can be found in documents cloud with name:
Unsubscribe cyclus from Sendgrid, MailChimp Spam and bounce
Checklijst
Default unsubscribe Form
Mailbox map
Blacklisted global
Bounce via sending providers
Spam reports via loopback sending providers
First method Exporting Gmail emails
First source of requests that is the mailbox itself where people forward and ask for their data to be deleted. Sometimes they don't want to click on a link or search or expect a complicated process but that's the fact quite a few unsubscribes end up in mailbox and in order not to manually process such requests one can automate Gmail in various ways.
Export data per label and then read this local mbox file read all messages and unsubscribe senders.
Exporting data might work even better than giving access to your account with script. Because then we can edit archive even better locally.
You can manage all your exports from this page. Create find where they are now etc..
https://takeout.google.com/
This method is good if you want to initiate it manually and it is more secure.
The other real time downloading of messages is not so good I think because you have to make your account less secure, there is no possibility to allow access only by IP, for example. When you open IMAP it is open to everyone and all that is left is your password.
Export is done almost immediately without delay as first mentioned hours or days.. That took 1 minute and I already had my files.. Imported emails via Drive with 1 label..
Now I have a zipped mbox file that I should now be able to read and write out all messages in this file so that they don't exist anymore.With this library it seems to be possible so we will test first and see how effective it is. For now this solution I think is the best option.. So how do I go about that just add all people in global unsubscribe list so they can leave all other services or work by list. People who have retired or complain can be completely removed from the system, others who have found work can still stay in system but put on inactive so that we ask back in 1 year if they are available.
https://docs.python.org/3/library/mailbox.html
Code will look something like this depending on what exactly is needed.
An example of script that can be used and further extended to a Django command or a similar script with a correct functional purpose.
# -*- coding: utf-8 -*-
#!/usr/bin/python
#
# Copyright (C) 2021 Web App Development
#
"""
"""
__author__ = 'sergej[A]dergatsjev[D]be (Sergej Dergatsjev)'
from django.db import models, transaction, IntegrityError
from django.core.management.base import BaseCommand
from django.template.defaultfilters import slugify
from crawler import utils
import re
from time import sleep
import sys, traceback
from newsletter.models import Subscriber
import mailbox
class Command(BaseCommand):
def handle(self, *args, **options):
"""
Entery point
"""
#import pdb;pdb.set_trace()
for message in mailbox.mbox("data/Unsubscribe.mbox"):
subject = message['subject']
from_email = message['From']
print("-----------------")
print(from_email)
if from_email.find('MAILER-DAEMON@secure-mail.be') != -1:
from_email = self.parse_from_email(message.as_string(), "for <([\w\-][\w\-\.]+@[\w\-][\w\-\.]+[a-zA-Z]{1,4})>")
print(from_email)
self.unsubscribe(from_email, "Undelivered")
elif from_email.find('<') != 1:
from_email = self.parse_from_email(from_email, "<([\w\-][\w\-\.]+@[\w\-][\w\-\.]+[a-zA-Z]{1,4})>")
self.unsubscribe(from_email, "Unsubscribe Mailbox")
else:
self.unsubscribe(from_email, "Unsubscribe Mailbox")
print(subject)
#email = utils.normalization_email(cols[0])
def parse_from_email(self, text, pattern):
pattern = re.compile(pattern)
result = re.findall(pattern, text)
for item in result:
if "vindazo.be" in item:
result.remove(item)
if len(result) > 0:
return result[0]
else:
return None
def unsubscribe(self, from_email, status):
if from_email:
subscriber = None
print(from_email)
try:
subscriber = Subscriber.objects.get(email=from_email, has_unsubscribe=False)
except:
try:
subscriber = Subscriber.objects.get(email=from_email)
print("All ready unsubscribed")
return None
except:
print("Email not found")
if subscriber:
subscriber.has_unsubscribe = True
subscriber.status = status
subscriber.save()
return from_email
Second example.. More complex case with different regular expressions to find "from mail"
# -*- coding: utf-8 -*-
#!/usr/bin/python
#
# Copyright (C) 2020 Web App Development EOOD
#
"""
Unsubscribe via profile
"""
import imaplib
import re
import requests
from datetime import datetime, date
import sys, traceback, os
from time import sleep
from spontaneousmail import utils
from django.core.management.base import BaseCommand
from spontaneousmail.models import SpontaneousProfile
import mailbox
class Command(BaseCommand):
verbose = True
timeout = 5
count = 0
mailbox = "data/Unsubscribe.mbox"
def handle(self, *args, **options):
"""
Unsubscribe via mailbox
"""
contacts = self.read_contacts()
count = 1
for contact in contacts:
print(count)
print(contact['email'])
count = count + 1
utils.unsubscribe(contact['email'], "Unsubscribe_mailbox")
def parse_from_email(self, text, pattern):
pattern = re.compile(pattern)
result = re.findall(pattern, text)
for item in result:
if ("vacaturemails.com" in item) or ("vacatures.today" in item):
result.remove(item)
if len(result) > 0:
return result[0]
else:
return None
def read_contacts(self):
"""
"""
#import pdb;pdb.set_trace()
contacts = []
count = 0
for message in mailbox.mbox(self.mailbox):
count = count + 1
subject = message['subject']
from_email = message['From']
print(count)
print(from_email)
if from_email.find('mailer-daemon') != -1:
# for <info@vacaturemails.com>
#import pdb;pdb.set_trace()
from_email = self.parse_from_email(message.as_string(), "for <([\w\-][\w\-\.]+@[\w\-][\w\-\.]+[a-zA-Z]{1,4})>")
if not from_email:
# Final-Recipient: rfc822; dd.ff@europe.com
from_email = self.parse_from_email(message.as_string(), "Final-Recipient: rfc822; ([\w\-][\w\-\.]+@[\w\-][\w\-\.]+[a-zA-Z]{1,4})")
elif from_email.find('<') != -1:
from_email = self.parse_from_email(from_email, "<([\w\-][\w\-\.]+@[\w\-][\w\-\.]+[a-zA-Z]{1,4})>")
if not from_email:
#import pdb;pdb.set_trace()
from_email = self.parse_from_email(message.as_string(), "To:.*([\w\-][\w\-\.]+@[\w\-][\w\-\.]+[a-zA-Z]{1,4})")
#import pdb;pdb.set_trace()
#print(subject)
if from_email:
contacts.append({"email":from_email})
else:
#import pdb;pdb.set_trace()
f = open("/tmp/emails/" + str(count), "w")
try:
f.write(message.as_string())
except:
traceback.print_exc(file=sys.stdout)
f.close()
return contacts
Unsubscribe Real Time via IMAP Gmail
Then the next method is about connection via IMAP and direct access to the correct label and processing all messages under a correct label such as Unsubscribe for example. People who manage mailbox move all mails under this label and script comes once a day to check if there is something to process and move in folder with all other ready messages in archive.
I don't recommend this way because it exposes your account to atteres from outside. But you can do it if you want. We've done it once for less important accounts too.
You will have an issues with login via IMAP.
b'[AUTHENTICATIONFAILED] Invalid credentials (Failure)'
python3 manage.py unsubscribe_imap
https://accounts.google.com/b/0/DisplayUnlockCaptcha
Occasionally Gmail will stop working and you can see a Postie error message that says something like “[ALERT] Please log in via your web browser”. This means you must log into the account Postie is trying to access via https://gmail.com where you will be asked to confirm a login from wherever your server is located.
If this doesn’t work try http://www.google.com/accounts/DisplayUnlockCaptcha
Turn off 2-Step Verification https://support.google.com/accounts/answer/1064203?hl=en If you are still having difficulty you may need to change the Google “Less secure apps” setting. Be sure you are only signed into the gmail account that postie will use then go to https://www.google.com/settings/security/lesssecureapps and make sure it is set to “Turn off”
While the wording on this page makes it sound dangerous, what it really means is “allow traditional authentication” which uses very high levels of encryption.
Finally check these pages for other options:
https://support.google.com/accounts/answer/6009563
https://support.google.com/mail/answer/14257
Code voorbeeld:
def login():
im = imaplib.IMAP4_SSL(INBOX_IMAP, 993)
im.login(INBOX_USER, INBOX_PASS)
return im
def unsubscribe(mailbox, im):
typ, data = im.search(None, 'ALL')
for num in data[0].split():
typ, msg = im.fetch(num, '(RFC822)')
unsubscribe_message(msg[0][1].decode("utf-8"), num, mailbox)
im = login()
unsubscribe("Unsubscribe", im)
# -*- coding: utf-8 -*-
#!/usr/bin/python
#
# Copyright (C) 2020 Web App Development EOOD
# Online Solutions Group BVBA
"""
Unsubscribe via profile
"""
import imaplib
import re
import requests
from datetime import datetime, date
import sys, traceback, os
from time import sleep
from spontaneousmail import utils
from django.core.management.base import BaseCommand
from spontaneousmail.models import SpontaneousProfile
INBOX_IMAP = 'imap.gmail.com'
INBOX_USER = ''
INBOX_PASS = ''
class Command(BaseCommand):
def handle(self, *args, **options):
"""
Unsubscribe via imap
"""
im = login()
unsubscribe("Unsubscribe", im)
def login():
im = imaplib.IMAP4_SSL(INBOX_IMAP, 993)
im.login(INBOX_USER, INBOX_PASS)
return im
def is_own(item):
OWN = ["vacaturemails.com", "vacatures.today"]
for own_email in OWN:
if item.find(own_email) != -1:
return True
return False
def get_to(message):
#import pdb;pdb.set_trace()
pattern = re.compile("From: [^\r]*\r\nTo: ([^\r]*)\r\nDate: ")
result = re.findall(pattern, message)
for item in result:
if not is_own(item):
return item
pattern = re.compile("\r\nTo:([^\r]*)\r\n")
result = re.findall(pattern, message)
for item in result:
if not is_own(item):
return item
def get_from(message):
pattern = re.compile("From: ([^\r]*)\r\n")
result = re.findall(pattern, message)
for item in result:
if not is_own(item):
return item
def clean(email):
if email.find("<") != -1:
pattern = re.compile("<([^>]*)>")
return re.findall(pattern, email)[0]
return email.strip()
def unsubscribe_remove(mailbox, im):
# Use onther map to move emails first
# Or if it realy should be deleted
print(im.select(mailbox))
typ, data = im.search(None, 'ALL')
for num in data[0].split():
typ, msg = im.fetch(num, '(RFC822)')
unsubscribe_message(msg[0][1], num, mailbox)
im.store(num, '+FLAGS', '\\Deleted')
im.expunge()
def remove_emails(mailbox, im):
# Like priviews but not used in this script only for exampole saved
print(im.select(mailbox))
typ, data = im.search(None, 'ALL')
import pdb;pdb.set_trace()
count = 0
max_count = 100
total = get_count(im)
while total > 0:
total = get_count(im)
count = count + 1
print(str(count))
if count == max_count:
im.expunge()
count = 1
try:
im.store(count, '+FLAGS', '\\Deleted')
except:
traceback.print_exc(file=sys.stdout)
im.expunge()
im.expunge()
def unsubscribe(mailbox, im):
# Usage like
#for mailbox in FOR_UNSUBSCRIBE:
# unsubscribe(mailbox, im)
print(im.select(mailbox))
import pdb;pdb.set_trace()
typ, data = im.search(None, 'ALL')
for num in data[0].split():
typ, msg = im.fetch(num, '(RFC822)')
unsubscribe_message(msg[0][1].decode("utf-8"), num, mailbox)
#im.expunge() deleted items from selected mailbox
def unsubscribe_message(message, num, label):
email = get_to(message)
if email == None:
email = get_from(message)
if email:
email = clean(email)
print(email)
# Here unsubscribe logic from database or via API url
#print(response)
utils.unsubscribe(email, "Unsubscribe_imap")
def get_count(im):
try:
typ, data = im.search(None, 'ALL')
except:
traceback.print_exc(file=sys.stdout)
im = login()
typ, data = im.search(None, 'ALL')
return len(data[0].split())
SendGrid MailChip unsubscribe automation
SendGrid and MailChip have the option to export their logs so you can export blocked and/or bounced messages from these services and unsubscribe automatically. They use CSV which is text format divided with koma.
Note they do not have the same format and are not in the same order so check the export order carefully.
Code example:
def handle(self, *args, **options):
"""
"""
files = [
'data/bounce/suppression_bounces.csv'
]
for file in files:
print(file)
f = open(file, "r")
items = f.readlines()
self.unsubsribe_emails(items)
def unsubsribe_emails(self, items):
import pdb;pdb.set_trace()
for item in items:
time.sleep(0.1)
item = item.split(',')[2]
try:
term = item.strip().lower()
print(term)
utils.unsubscribe(term, "Bounce")
except:
traceback.print_exc(file=sys.stdout)
More examples see in intern commands of mailing application in CRM.
For example:
mailing/management/commands/importsendgridspam.py
mailing/management/commands/unsubscribe_suppression.py
mailing/management/commands/importsendgridbounce.py
mailing/management/commands/collect_suppression.py
mailing/management/commands/copyemailsfromstatus.py
If you start to unsubscribe automatically then look very carefully at status and message because some messages mean nothing but blocked and that is per domain or even message level and bounce mailbox is still OK but sender is not OK and he is blocked and this bounce actually belongs in category blocked en not bounce.. Bounce is when mailbox is closed or overquite. So look good, such message from from skynet.be provider is about your blacklisted IP, Domain or message by skynet provider..
550 #5.1.0 Address rejected.
When you filter this messages and unsubscribe it from list, unsubscribe it not on global level but only per domain and message.
To read messages in mail log from big logfile about 50 GB.. For example, if you have a configuration that contains all baskets or even years, it could be that it is not readable with VIM. Then you can cut and read the last 100K from this file.
du -h /var/log/mail.log
50G /var/log/mail.log
tail -n 100000 /var/log/mail.log > /tmp/log
Unsubscribe via Firebase form
We have several ways to accomplish this task. We can use the REST API and lib which was made 5 years ago.
https://firebase.google.com/docs/database/rest/start
https://github.com/thisbejim/Pyrebase
Or even older like
https://github.com/ozgur/python-firebase
So it's not recommended that we write new applications and use Realtime Database for my purposes, I've found Firestore Database works much better, and I'm trying to limit or eliminate this old use altogether in the future.
Next option is also an export of data. If for example you only need to perform this unsubscribe operation 1 time per month then you can export data and unsubscribe.
But we will automate immediately in a normal way because I think this application will be used for a long time and proper automation is needed from the start.
Pyrebase was written for python 3 and will not work correctly with python 2.
The first step is the configuration.
import pyrebase
config = {
"apiKey": "AIzaSyAfGNahILALZrp25B_-l97-dfdfdf",
"authDomain": "vacatures.firebaseapp.com",
"databaseURL": "https://vacatures.firebaseio.com",
"storageBucket": "vacatures.appspot.com"
"serviceAccount": "path/to/serviceAccountCredentials.json"
}
firebase = pyrebase.initialize_app(config)
In other words you can find all config information in Web or App configs.
https://firebase.google.com/docs/projects/api-keys
var firebaseConfig = {
apiKey: "AIzaSyAfGNahILALZsdfsdfllSdd-dfdf",
authDomain: "vacature.firebaseapp.com",
databaseURL: "https://vacature.firebaseio.com",
projectId: "vacature",
storageBucket: "vacatures.appspot.com",
messagingSenderId: "4455454",
appId: "1:479857347200:web:343434",
measurementId: "G-DFDF4545"
};
But if we need full Admin access we can still
"serviceAccount": "path/to/serviceAccountCredentials.json"
In this way we can read data that is restricted with rights for other users.
With this wrapper you can address a number of services for example auth service.
https://github.com/thisbejim/Pyrebase#authentication
For example, it is possible to create or delete a user automatically.
So if your unsubscribe logic includes removing users then you can do it automatically via this wrapper reference.
At the moment we will only remove people who have unsubscribed from mailing and leave users unchanged.
Example
class Command(BaseCommand):
"""
Unsubscriebe emails from firebase database
"""
def handle(self, *args, **options):
"""
"""
firebase = pyrebase.initialize_app(config)
db = firebase.database()
contacts = db.child("unsubscribe").get()
#import pdb;pdb.set_trace()
for contact in contacts.each():
self.unsubsribe_emails(contact.val())
def unsubsribe_emails(self, item):
#import pdb;pdb.set_trace()
try:
term = item['email'].strip().lower()
print(term)
utils.unsubscribe(term, "Unsubscribe form")
except:
traceback.print_exc(file=sys.stdout)
Comments
Post a Comment