pymongo

MongoDB Aggregation with $sample very slow

拜拜、爱过 提交于 2020-01-24 09:20:28
问题 There are many ways to select random document from a mongodb collection (as discussed in this answer). Comments point out that with mongodb version >= 3.2 then using $sample in the aggregation framework is preferred. However, on a collection with many small documents this seems to extremely slow. The following code uses mongoengine to simulate the issue and compare it to the "skip random" method: import timeit from random import randint import mongoengine as mdb mdb.connect("test-agg") class

Cannot connect to mongodb instance from docker: Connection refused

半世苍凉 提交于 2020-01-23 12:46:09
问题 I am using docker-compose to create a multi-container environment where I have one mongodb instance and two python applications. The problem is, the first application is able to establish a connection to mongodb whereas the second application fails with the following error: File "/usr/local/lib/python2.7/site-packages/pymongo/mongo_client.py", line 377, in __init__ notification_1 | raise ConnectionFailure(str(e)) notification_1 | pymongo.errors.ConnectionFailure: [Errno -2] Name or service

Sanitizing inputs to MongoDB

一笑奈何 提交于 2020-01-23 08:27:41
问题 I'm writing a REST interface to a MongoDB database program, and I'm trying to implement search functionality. I'd like to expose the entire MongoDB interface. I have two questions, really, but they're related so I'm putting them in one post: Is it safe to decode untrusted JSON with the Python json module, or is it like the pickle module in that it could allow arbitrary code execution? Is it safe to pass that decoded JSON to the PyMongo find() function? 回答1: Python's JSON module should be safe

MongoDB, Flask, aggregate with $unwind query

与世无争的帅哥 提交于 2020-01-23 07:55:06
问题 Context I have a Flask application connected to MongoDB using pymongo. Currently the database contains data about runs in this format: { "_id": { "$oid": "5df031cec687bf2b4c4349b9" }, "run_number": "1", "frames": [{ "frame_number": 1, "data": { "brake": "0.1", "steer": "0.4", "throttle": "0.6" } }, { "frame_number": 2, "data": { "brake": "0.2", "steer": "0.8", "throttle": "0.6" } }, { "frame_number": 3, "data": { "brake": "0.6", "steer": "0.2", "throttle": "0.1" } }] } I am able to retrieve

PyMongo/Mongoengine equivalent of mongodump

喜欢而已 提交于 2020-01-23 05:51:24
问题 Is there an equivalent function in PyMongo or mongoengine to MongoDB's mongodump ? I can't seem to find anything in the docs. Use case: I need to periodically backup a remote mongo database. The local machine is a production server that does not have mongo installed, and I do not have admin rights, so I can't use subprocess to call mongodump . I could install the mongo client locally on a virtualenv, but I'd prefer an API call. Thanks a lot :-). 回答1: For my relatively small small database, I

PyMongo vs MongoEngine for Django

让人想犯罪 __ 提交于 2020-01-22 05:15:09
问题 For one of my projects I prefered using Django+Mongo . Why should I use MongoEngine , but not just PyMongo ? What are advantages? Querying with PyMongo gives results that are allready objects , aren't they? So what is the purpose of MongoEngine? 回答1: I assume you have not read the MongoEngine claim. MongoEngine is a Document-Object Mapper (think ORM, but for document databases) for working with MongoDB from Python. This basically say it all. In addition: your claim that Pymongo would deliver

python MongoDB

给你一囗甜甜゛ 提交于 2020-01-17 15:31:15
MongoDB官网下载地址 https://www.mongodb.com/download-center/community 1 连接 安装pymongo pip install pymongo 连接MongoDB from pymongo import MongoClient from datetime import datetime conn = MongoClient ( 'localhost' , 27017 ) db = conn . Student table = db . student #删除所有age大于10的数据 table . delete_many ( { "age" : { "$gt" : 10 } } ) stu = [ { "name" : "1" , "age" : 30 } , { "name" : "2" , "age" : 30 } , { "name" : "3" , "age" : 30 } , { "name" : "4" , "age" : 30 } ] table . insert_many ( stu ) table . insert_one ( { "name" : "chen" , "age" : 29 } ) table . update_one ( { "name" : "chen" } , { "$set" : {

python爬虫备忘录

柔情痞子 提交于 2020-01-17 06:08:28
我都不知道多久没有发过博文了,伴随着毕业的到来,论文和实习,都一起到来了,可能我以后也很少发布编程类的文章了,更多的将会注重于网络安全文章的发布了,windowsAPI的文章将会逐渐从我的博文中删除,以后将会不定期更新webdirscan,weblogon_brust等的python技术文章,二进制或者手机APP类型的,很感叹自己一路走过来,学习了不少的知识,也遇到过很多大佬,向他们学习了一些知识,到如今,我还是觉得脚踏实地的走比较靠谱。之后我会陆续更新我的开源webdirscan软件,开源一些信息收集的小工具。 爬虫环境配置 selenium 描述:模拟浏览器访问,有界面 安装: pip3 install selenium 基本使用: import selenium from selenium import webdriver driver=webdriver.Chrome() chromedriver 描述:谷歌驱动 安装:pip install chromedriver 基本使用: import selenium from selenium import webdriver driver=webdriver.Chrome() phantomjs 描述:模拟浏览器访问,无界面 安装: 下载https://phantomjs.org/download 配置: export

centos7离线安装Python3和pymongo

拈花ヽ惹草 提交于 2020-01-16 21:08:43
目录 离线安装Python3 离线安装setuptools 离线安装pymongo 由于我要在虚拟机上安装python3和pymongo,而虚拟机无法联网,因此需要先下载好安装包再上传到虚拟机上,进行离线安装。 离线安装Python3 链接: Python官网 . 点击download,选择linux并下载相应的版本,下载完成后解压并上传到centos7 下载一些python和pymongo需要的库,如zlib、libfi等,下载完成后解压并上传到centos7 下载链接 . 接下来先安装rpm包,输入如下命令 rpm -ivh XXX.rpm 如果出现下图所示情况,则进行强制安装 rpm -ivh --nodeps XXX.rpm --force 安装完成后,使用如下命令安装Python ./configure prefix=/usr/local/python3 该命令表示在/usr/local/下创建一个叫python3的文件夹,把python3安装在这里,若没有指定路径,则默认安装在/usr/local/下 在输入如下命令 make make install 若遇到权限问题,可先修改Python目录的权限,进入python3安装包文件夹,输入如下命令 chmod 777 -R …/Python-3.5.2/ 接下来使用软链接将Python3链接到Python

How do I perform the SQL Join equivalent in PyMongo? Or more specific call a Pymongo collection object in BSON code?

烂漫一生 提交于 2020-01-16 18:31:11
问题 I'm trying to perform a SQL Join equivalent in pymongo like so: http://blog.knoldus.com/2013/02/03/joins-now-possible-in-mongodb/ The thing is, I am stuck as bson cannot encode collection objects. bson.errors.InvalidDocument: Cannot encode object: Collection(Database(MongoClient('localhost', 27017), 'Application'), 'Products') All relevant code: class Delivery(Repository): COLLECTION_ATTRIBUTE = 'deliveriesCollection' def __init__(self): Repository.__init__(self, self.COLLECTION_ATTRIBUTE)