问题
I'm using Google App Engine python 3.7 standard and i'm trying to group related request log entries. According to the Writing Application Logs documentation, I should:
Set the trace identifier in the LogEntry trace field of your app log entries. The expected format is projects/[PROJECT_ID]/traces/[TRACE_ID]
Where/How should use LogEntry?
The Stackdriver Logging documentation doesn't show how it's possible. Am I missing something?
Code examples would be much appreciated.
[UPDATE] Following Duck Hunt Duo advice, I tried the following, without any success:
trace_id = request.headers.get('X-Cloud-Trace-Context', 'no_trace_id').split('/')[0]
client = logging.Client()
logger = client.logger('appengine.googleapis.com%2Fstdout') # Not shown
# logger = client.logger('projects/{}/logs/stdout'.format(GOOGLE_CLOUD_PROJECT)) # error
# logger = client.logger('projects/{}/logs/appengine.googleapis.com%2Fstdout'.format(GOOGLE_CLOUD_PROJECT)) # error
logger.log_text('log_message', trace=trace_id)
The log doesn't appear in the GAE service log web console
回答1:
You might want to take a look at an answer I provided here.
(This answer addresses how to add logging severity to Cloud Functions logs written into Stackdriver, but the basic workflow is the same)
Quoting it:
[...], you can still create logs with certain severity by using the Stackdriver Logging Client Libraries. Check this documentation in reference to the Python libraries, and this one for some usage-case examples.
Notice that in order to let the logs be under the correct resource, you will have to manually configure them, see this list for the supported resource types. As well, each resource type has some required labels that need to be present in the log structure.
Edit:
Updating the previous answer with an example for App Engine:
from google.cloud import logging
from google.cloud.logging.resource import Resource
from flask import Flask
app = Flask(__name__)
@app.route('/')
def logger():
log_client = logging.Client()
log_name = 'appengine.googleapis.com%2Fstdout'
res = Resource( type='gae_app',
labels={
"project_id": "MY-PROJECT-ID",
"module_id": "MY-SERVICE-NAME"
})
logger = log_client.logger(log_name)
logger.log_struct({"message": "message string to log"}, resource=res, severity='ERROR') # As an example log message with a ERROR warning level
return 'Wrote logs to {}.'.format(logger.name)
By using this code as example, and changing the resource type of the log to appengine.googleapis.com%2Fstdout should work, and change the Resource fields to be the same as in the gae_app labels described in here.
回答2:
This is my basic solution:
trace_id = request.headers.get('X-Cloud-Trace-Context', 'no_trace_id').split('/')[0]
trace_str = "projects/{}/traces/{}".format(os.getenv('GOOGLE_CLOUD_PROJECT'), trace_id)
log_client = logging.Client()
# This is the resource type of the log
log_name = 'stdout'
# Inside the resource, nest the required labels specific to the resource type
labels = {
'module_id': os.getenv('GAE_SERVICE'),
'project_id': os.getenv('GOOGLE_CLOUD_PROJECT'),
'version_id': os.getenv('GAE_VERSION')
}
res = Resource(type="gae_app",
labels=labels,
)
logger = log_client.logger(log_name)
logger.log_text("MESSAGE_STRING_TO_LOG", resource=res, severity='ERROR', trace=trace_str)
After it was working, I wrapped it in a file so it would work similarly to Google's logger for python2.7 .
Here is my_gae_logging.py:
import logging as python_logging
import os
from flask import request
from google.cloud import logging as gcp_logging
from google.cloud.logging.resource import Resource
# From GCP logging lib for Python2.7
CRITICAL = 50
FATAL = CRITICAL
ERROR = 40
WARNING = 30
WARN = WARNING
INFO = 20
DEBUG = 10
NOTSET = 0
_levelNames = {
CRITICAL: 'CRITICAL',
ERROR: 'ERROR',
WARNING: 'WARNING',
INFO: 'INFO',
DEBUG: 'DEBUG',
NOTSET: 'NOTSET',
'CRITICAL': CRITICAL,
'ERROR': ERROR,
'WARN': WARNING,
'WARNING': WARNING,
'INFO': INFO,
'DEBUG': DEBUG,
'NOTSET': NOTSET,
}
def get_trace_id():
trace_str = None
try:
trace_id = request.headers.get('X-Cloud-Trace-Context', 'no_trace_id').split('/')[0]
trace_str = "projects/{project_id}/traces/{trace_id}".format(
project_id=os.getenv('GOOGLE_CLOUD_PROJECT'),
trace_id=trace_id)
except:
pass
return trace_str
class Logging:
def __init__(self):
self._logger = None
@property
def logger(self):
if self._logger is not None:
return self._logger
log_client = gcp_logging.Client()
# This is the resource type of the log
log_name = 'appengine.googleapis.com%2Fstdout'
# Inside the resource, nest the required labels specific to the resource type
self._logger = log_client.logger(log_name)
return self._logger
@property
def resource(self):
resource = Resource(
type="gae_app",
labels={
'module_id': os.getenv('GAE_SERVICE'),
'project_id': os.getenv('GOOGLE_CLOUD_PROJECT'),
'version_id': os.getenv('GAE_VERSION')
}
)
return resource
def log(self, text):
text = str(text)
self.logger.log_text(text, resource=self.resource, trace=get_trace_id())
def debug(self, text):
text = str(text)
self.logger.log_text(text, resource=self.resource, severity=_levelNames.get(DEBUG), trace=get_trace_id())
def info(self, text):
text = str(text)
self.logger.log_text(text, resource=self.resource, severity=_levelNames.get(INFO), trace=get_trace_id())
def warning(self, text):
text = str(text)
self.logger.log_text(text, resource=self.resource, severity=_levelNames.get(WARNING), trace=get_trace_id())
def warn(self, text):
return self.warning(text)
def error(self, text):
text = str(text)
self.logger.log_text(text, resource=self.resource, severity=_levelNames.get(ERROR), trace=get_trace_id())
def critical(self, text):
text = str(text)
self.logger.log_text(text, resource=self.resource, severity=_levelNames.get(CRITICAL), trace=get_trace_id())
if os.getenv('GAE_VERSION'): # check if running under gcp env
logging = Logging()
else:
# when not running under gcp env, use standard python_logging
logging = python_logging
Usage:
from my_gae_logging import logging
logging.warn('this is my warning')
回答3:
The Stackdriver Logging Client Library can be used to achieve this. The logger.log_text function sends a LogEntry object to the API. Example:
from google.cloud import logging
client = logging.Client()
logger = client.logger('appengine.googleapis.com%2Fstdout')
logger.log_text('log_message', trace=trace_id)
The trace_id should be retrieved from the request headers as the docs mention. The method of doing this will depend on how you're serving requests, but in Flask for example it would be simple as trace_id = request.headers['X-Cloud-Trace-Context'].split('/')[0]
回答4:
Using the AppEngineHandler from Google Cloud Logging provides much of the infrastructure. This allows attaching to the python logging module, so that a standard logging import works.
Setting this up is straightforward enough:
# Setup google cloud logging.
import logging
import google.cloud.logging # Don't conflict with standard logging
from google.cloud.logging.handlers import AppEngineHandler, setup_logging
client = google.cloud.logging.Client()
handler = AppEngineHandler(client, name='stdout')
logging.getLogger().setLevel(logging.INFO)
setup_logging(handler)
The documentation at https://googleapis.dev/python/logging/latest/usage.html#cloud-logging-handler suggests very similar, but instead of using the AppEngineHandler uses the "CloudLoggingHandler". It also states that the "AppEngineHandler" is for the flexible environment, but this works in the standard python3 environment.
来源：https://stackoverflow.com/questions/56113604/how-to-group-related-request-log-entries-gae-python-3-7-standard-env