ErrorShip

Send exceptions to datadog.


Home        Pricing        Get License Key        Privacy & Security       

Are you tired of looking at metrics in datadog and then switching over to another website to track your applications exceptions?

errorShip is a python library that sends exceptions/errors generated by your application to your datadog account.
The errorship SDK source code can be found at gitlab.com/errorship

 


1. Installation:


pip install errorship
                    
Next, obtain a license key from errorship.com; this will enable you to be able to configure the erroship log handler.

errorship does not send data directly to datadog, rather it sends the data to a datadog agent which will then send the data to datadog. You thus need to have a datadog agent running for errorship to work correctly.
For more information on how to correctly configure and run a datadog agent, consult the datadog agent documentation

2. Usage:
2.1 Simple Usage:
2.1.1 In regular python:
Add the following piece of code at a place where it can be executed very early at the startup of your application. Such a place can be in an __init__.py file or at the top of your main.py


import errorship

errorship.setup(
    datadog_agent_host="localhost",
    datadog_agent_port=8125,
    errorship_license_key="MyErrorshipLicenseKey",
    tags={
        # this tags will be added to the errors/exceptions
        # that are sent to datadog
        "env": "production",
        "project": "accounting",
        "deployment_version": "v12.56"
    }
)
                    
And then, you can continue logging in your application as before. ie, you do not have to change any other place in your application and errorship will just work.

# your application
import logging

try:
    database ={}
    name = database["James"]
    print("The user is: ", name)
except Exception as e:
    logging.exception("database error. error={0}".format(e))
                    
2.1.2 In django:
Add the following in your settings.py file;

import errorship

errorship.setup(
    datadog_agent_host="localhost",
    datadog_agent_port=8125,
    errorship_license_key="MyErrorshipLicenseKey",
    tags={
        # this tags will be added to the errors/exceptions
        # that are sent to datadog
        "env": "production",
        "project": "accounting",
        "deployment_version": "v12.56"
    }
)
               

2.1.3 In flask:
Add the following in your app.py file;

from flask import Flask
import errorship

errorship.setup(
    datadog_agent_host="localhost",
    datadog_agent_port=8125,
    errorship_license_key="MyErrorshipLicenseKey",
    tags={
        # this tags will be added to the errors/exceptions
        # that are sent to datadog
        "env": "production",
        "project": "accounting",
        "deployment_version": "v12.56"
    }
)
app = Flask(__name__)
                

2.1.4 In Celery:
Add the following in your app.py file;

from celery import Celery
import errorship

errorship.setup(
    datadog_agent_host="localhost",
    datadog_agent_port=8125,
    errorship_license_key="MyErrorshipLicenseKey",
    tags={
        # this tags will be added to the errors/exceptions
        # that are sent to datadog
        "env": "production",
        "project": "accounting",
        "deployment_version": "v12.56"
    }
)
app = Celery()
                


2.2 Advanced Usage:
2.2.1 In regular python:
It is possible to add the errorship handler to just a particular logger;

import logging
import errorship

# setup logger
logger = logging.getLogger("MyApiLogger")
level = "INFO"
handler = errorship.Handler(
    datadog_agent_host="localhost",
    datadog_agent_port=8125,
    errorship_license_key="MyErrorshipLicenseKey",
    tags={
    "env": "production",
    "project": "accounting",
    "someKey": "someOtherValue"
    },
)
handler.setFormatter(logging.Formatter("%(message)s"))
handler.setLevel(level)
logger.addHandler(handler)
logger.setLevel(level)

# use logger
logger.info("send email start")
try:
    messages = {}
    _ = messages["first"]  # keyError
except Exception as e:
    logger.exception("send_email_error. error={0}".format(e))

In this case we are only adding the errorship handler to the logger named MyApiLogger
2.2.2 In django:
Add the following in your settings.py file;

LOGGING = {
    "version": 1,
    "disable_existing_loggers": False,
    "handlers": {
        # Integrate Django logging with errorship
        "errorship": {
            "level": "ERROR",
            "class": "errorship.Handler",
            "datadog_agent_host": "localhost",
            "datadog_agent_port": 8125,
            "errorship_license_key": "MyErrorshipLicenseKey",
            "tags": {
                "env": "production",
                "project": "accounting",
                "region": "us-west-1",
                "someKey": "someOtherValue",
            },
        },
        "console": {"level": "INFO", "class": "logging.StreamHandler",},
    },
    "loggers": {
        "django": {"handlers": ["errorship", "console"], "level": "INFO"},
        "myApp": {"handlers": ["errorship", "console"], "level": "INFO"},
    },
    "root": {"handlers": ["errorship", "console"], "level": "WARNING",},
}
                        

This configures two handlers; errorship and console. The errorship handler is set to handle all logs of loglevel ERROR and above, whereas the console handler deals with logs of loglevel INFO and above.
When the errorship handler deals with logs it will send any errors/exceptions to datadog, whereas the console log handler will stream the logs to stdout
For more information on how to configure django logging consult the django documentation
2.2.3 In flask
Add the following in your app.py file;

from flask import Flask
from logging.config import dictConfig

# See flask documentation:
# http://flask.palletsprojects.com/logging/#basic-configuration
dictConfig(
    {
        "version": 1,
        "formatters": {"default": {"format": "%(module)s: %(message)s"}},
        "handlers": {
            "wsgi": {
                "class": "logging.StreamHandler",
                "stream": "ext://flask.logging.wsgi_errors_stream",
                "formatter": "default",
            },
            # Integrate Flask logging with errorship
            "errorship": {
                "level": "ERROR",
                "class": "errorship.Handler",
                "datadog_agent_host": "localhost",
                "datadog_agent_port": 8125,
                "errorship_license_key": "MyErrorshipLicenseKey",
                "tags": {
                    "env": "canary",
                    "project": "accounting",
                    "region": "us-west-1",
                    "someKey": "someOtherValue",
                },
            },
        },
        "root": {"level": "INFO", "handlers": ["errorship", "wsgi"]},
    }
)
app = Flask(__name__)
                                        

For more information on how to configure flask logging consult the flask documentation

3. Step by step tutorial:
We are going to create a small application that checks the status of various popular websites and reports whether they are up or not. The application will be called uptime_checker
The application is going to be instrumented with errorShip so that any errors encountered can be shipped to datadog.

Lets start by creating a directory for our application.

mkdir /tmp/uptime_checker && \
cd /tmp/uptime_checker
                    
Next we install errorship and confirm that it has installed succesfully.

pip install errorship && \
pip freeze
    errorship==0.0.1 # note: your version number may vary
                    
In order to use errorship we need a license_key. Visit errorship.com and get a license_key for your project.

The errorship library sends any errors/exceptions to a datadog agent that runs in your infrastructure. This way, errorship does not transmit any of your application's errors or source code to our servers.
errorship can use the same datadog agent that you are already using to ship metrics to datadog. Consult the datadog agent documentation on how to get one running if you haven't already.
An easy way to run the datadog agent would be to do so via docker;

docker run -it \
    -p 8125:8125/udp \
    -e DD_DOGSTATSD_PORT=8125 \
    -e DD_BIND_HOST=0.0.0.0 \
    -e DD_DOGSTATSD_NON_LOCAL_TRAFFIC=true \
    -e DD_API_KEY=${Replace-with-your-datadog-API-key} \
    datadog/agent:latest

# or if you use datadog's european region;
docker run -it \
    -p 8125:8125/udp \
    -e DD_SITE=datadoghq.eu \
    -e DD_DOGSTATSD_PORT=8125 \
    -e DD_BIND_HOST=0.0.0.0 \
    -e DD_DOGSTATSD_NON_LOCAL_TRAFFIC=true \
    -e DD_API_KEY=${Replace-with-your-datadog-API-key} \
    datadog/agent:latest
                    

With that out of the way, lets now write the code for our application, all of it will be in one file; myApp.py

# /tmp/uptime_checker/myApp.py
import errorship

errorship.setup(
    datadog_agent_host="localhost",
    datadog_agent_port=8125,
    errorship_license_key="bq45ubqnmmjcqrklmv30",
    tags={
        # this tags will be added to the errors/exceptions
        # that are sent to datadog
        "service": "uptime_checker",
        "env": "dev",
    },
)

def main():
    import logging
    import requests  # pip install requests

    logging.info("application_start")
    websites = [
        "https://www.google.com/",
        "https://errorship.com/",
        "httTTTps://www.wikipedia.org/",
    ]

    WEBSITE_STATUS = {}
    for url in websites:
        try:
            response = requests.get(url)
            WEBSITE_STATUS.update({url: "UP"})
        except requests.RequestException as e:
            logging.error("errror. unable to fetch website; {0}".format(url))
            WEBSITE_STATUS.update({url: "DOWN"})

    print("The status of the websites is:")
    print(WEBSITE_STATUS)

if __name__ == "__main__":
    main()
We can now run our application;

python /tmp/uptime_checker/myApp.py 
And the output will be;

The status of the websites is:
{
    'https://www.google.com/': 'UP',
    'https://errorship.com/': 'UP',
    'httTTTps://www.wikipedia.org/': 'DOWN'
}
One of the websites, httTTTps://www.wikipedia.org is reported as been down. This is because, it has an invalid scheme; httTTTps
Thus trying to fetch it should raise an error. Since we caught the error and logged it, errorShip should have sent the error details to our datadog account.
Let's check; Navigate to your datadog account's event stream. It is usually located at https://app.datadoghq.com/event/stream

tutorial main error datadog event stream

That is how the errors, their stacktraces and other metadata are displayed in the datadog event stream.

4. Accessing errors in datadog:
The errors/exceptions are sent to the datadog event stream.
The eventstream has powerful full text search, and the full text search looks inside the event text, title, tags, users who commented on the event, host names, and devices tied to the event.

This full text search is available even for the errors that are sent by errorShip
Taking the example from the previous section of 3. Step by step tutorial; we can search for errors/exceptions using the provided tags, eg; tags:service:uptime_checker:

tutorial search errors by tag

We can also search for any text that is inside the error/exception, stack trace or metadata. For example, we can search using the text; wikipedia since it appears in the stacktrace;

tutorial search errors by any text


There is a lot more you can do with the errors once they are in the datadog event stream. For example,
  • you can filter; where you target specific event properties using these prefixes
  • you can use the Datadog event query language
  • you can use @notifications in the event stream
and much more.
Check out the datadog eventstream documentation for more information.

.
.


You can direct any queries/feedback to: support@errorship.com

.