Im trying to convert big number, a really big number more then 1000 digits, I want it to be hex 128bit .
Tried using this :
def tohex(val, 128):
return hex((val + (1 << 128)) % (1 << 128))
but with big numbers all I get is something like this :
0x0L
hex(x) function does not work also.
in the end I need to convert a really big number in to 32digits hex (128bit) any suggestions guys?
I am trying to learn python for Machine Learning and I just copied an example from the test book to my IDEL program. Here is the code, copied from here
from numpy import *
import operator
def createDataSet():
group = array([[1.0,1.1], [1.0,1.0], [0,0], [0,0.1]])
labels = ['A','A','B','B']
return group, labels
def classify0(inX, dataSet, labels, k):
dataSetSize = dataSet.shape[0]
diffMat = tile(inX, (dataSetSize,1)) - dataSet
sqDiffMat = diffMat ** 2
sqDistances = sqDiffMat.sum(axis = 1)
distances = sqDistances ** 0.5
sortedDistIndicies = distances.argsort()
# Voting with lowest k distances
classCount={}
for i in range(k):
voteIlabel = labels[sortedDistIndicies[i]]
classCount[voteIlabel] = classCount.get(voteIlabel, 0) + 1
# Decompose into a list of tuples and sort by second item
sortedClassCount = sorted(classCount.iteritems(),
key=operator.itemgetter(1), reverse=True)
return sortedClassCount[0][0]
group, labels = createDataSet()
classify0([0,0], group, labels, 3)
Then I compel it in IDLE but I am keep getting the following error message in Python Shell. (I am using version 3.4.3)
The error message is:
Traceback (most recent call last):
File "/Users/Python/KNN.py", line 32, in <module>
classify0([0,0], group, labels, 3)
File "/Users/Python/KNN.py", line 26, in classify0
sortedClassCount = sorted(classCount.itemgetter(),
AttributeError: 'dict' object has no attribute 'itemgetter'
I just don't understand why this is not working...and I don't even see where this 'dict' object is...Please help!
I'm running a small Django server on Ubuntu with virtualenv, apache2 and mod_wsgi.
I have virtualenv created with python3 on both windows and linux. pip --version shows:
(env)lbr@irma-fr:/var/www/irmaweb/irmaweb$ pip --version
pip 7.0.3 from /var/www/irmaweb/env/lib/python3.4/site-packages (python 3.4)
It works fine on windows, but when deploying on my Ubuntu server i get the following error:
The error is because Django create a new empty DB in /var/www instead of using the DB created by ./manage.py migrate in my project root:
(env)lbr@irma-fr:/var/www/irmaweb/irmaweb$ ls -al /var/www
total 20
drwxrws--- 5 lbr www-data 4096 Jul 1 05:58 .
drwxr-xr-x 12 root root 4096 Jun 26 09:53 ..
-rw-r--r-- 1 www-data www-data 0 Jul 1 05:58 dev.db
But I think the root of the issue is because Django get called from python2 as shown in the screenshot instead of getting called by python3 as my virtualenv is configured for.
Here is below my apache2 configuration:
(env)lbr@irma-fr:/var/www/irmaweb/irmaweb$ cat /etc/apache2/sites-available/000-default.conf
<VirtualHost *:80>
WSGIDaemonProcess irmaweb python-path=/var/www/irmaweb/irmaweb:/var/www/irmaweb/env/lib/python3.4/site-packages
WSGIProcessGroup irmaweb
WSGIScriptAlias / /var/www/irmaweb/irmaweb/irmaweb/wsgi.py
Alias /site_media/static/ /var/www/irmaweb/irmaweb/irmaweb/site_media/static/
So I guess I'm missing something! how to tell apache/wsgi to call python from my virtualenv folder:
(env)lbr@irma-fr:/var/www/irmaweb$ ls -al env/bin/python*
lrwxrwxrwx 1 lbr www-data 7 Jun 30 15:19 env/bin/python -> python3
-rwxrwxr-x 1 lbr www-data 4061272 Jun 30 15:19 env/bin/python3
lrwxrwxrwx 1 lbr www-data 7 Jun 30 15:19 env/bin/python3.4 -> python3
Instead of using the system python from /usr/bin ?
The opencv works fine when doing other things. It can can open images and show images. But it can't open a video.
The code I'm using to open a video is as below
import cv2
cap = cv2.VideoCapture("MOV_0006.mp4")
while True:
ret, frame = cap.read()
cv2.imshow('video', frame)
if cv2.waitKey(1) & 0xff == ord('q'):
break
cap.release()
cv2.destroyAllWindows()
But when executing, it outputs error message like below
[h264 @ 0x1053ba0] AVC: nal size 554779904
[h264 @ 0x1053ba0] AVC: nal size 554779904
[h264 @ 0x1053ba0] no frame!
My vlc and mplayer can play this video, but the opencv can't.
I have installed x264 and libx264-142 codec package. (using sudo apt-get install)
My version of ubuntu is 14.04 trusty.
I'm not sure is it a codec problem or not?
I have rebuilt opencv either with WITH_UNICAP=ON or with WITH_UNICAP=OFF, but it doesn't affect the problem at all. The error message never changes.
and what is the data type of that result, does that data type match the function open('xx.jpg').read()
my purpose is to read image from lmdb and then match my code (that code is written for data type of open('xx.jpg').read() ),so I want to get the same data type from lmdb if I can
i am using yii2 and guzzle version 4 when for api pyhton Django 1.5 when i am trying to post data with postman is working but when i am trying with php is saying bad request 404
error is GuzzleHttp\Exception\ClientException
v1/jobs/ [status code] 400 [reason phrase] BAD REQUEST
post man format data working data format
url -vm.xxx.com/v1/jobs/
{
"job_position":"Test Postion",
"eligibility_course_id":["1","2","3","4"],
"eligibiltiy_course_ids":["1","2","3","4"],
"eligibility_branch_id":["5","6","7","8"],
"eligibiltiy_skill_ids":["10","23","24"],
"eligibiltiy_sublocation_ids":["10","23","24"],
"location":["26","30","2","3"],
"job_category":["1","4","7"],
"sms_category":"1",
"posting_date":"31\/07\/2014",
"expiry_date":"31\/08\/2014",
"job_type_id":"1",
"job_type_other":"",
"hiring_process":["1"],
"job_description":"Test Description<\/p>",
"company_name":"Test Company name",
"company_profile":"Test company profile<\/p>",
"company_url":"",
"seo_title":"Test Seo Title",
"seo_description":"Test Seo Description",
"seo_keyword":"Test seo Keyward",
"response":"1","how_to_apply":"",
"terms_agreement":"1",
"specify_terms_agreement":"",
"employer_id":"1008369",
"button_submitForm":"Post Job",
"posted_by_id":"114",
"template_name":"job_description"
}
with php code method -1
$arr=[
"job_position" => "Test Postion",
"eligibility_course_id" => ["1","2","3","4"],
"eligibiltiy_course_ids" => ["1","2","3","4"],
"eligibility_branch_id" => ["5","6","7","8"],
"eligibiltiy_skill_ids" => ["10","23","24"],
"eligibiltiy_sublocation_ids" => ["10","23","24"],
"location" => ["26","30","2","3"],
"job_category" => ["1","4","7"],
"sms_category" => "1",
"posting_date" => "31\/07\/2014",
"expiry_date" => "31\/08\/2014",
"job_type_id" => "1",
"job_type_other" => "",
"hiring_process" => ["1"],
"job_description" => "Test Description<\/p>",
"company_name" => "Test Company name",
"company_profile" => "Test company profile<\/p>",
"company_url" => "",
"seo_title" => "Test Seo Title",
"seo_description" => "Test Seo Description",
"seo_keyword" => "Test seo Keyward",
"response" => "1","how_to_apply" => "",
"terms_agreement" => "1",
"specify_terms_agreement" => "",
"employer_id" => "1008369",
"button_submitForm" => "Post Job",
"posted_by_id" => "114",
"template_name" => "job_description"
];
$client = new Client();
$apiurl=Yii::$app->params['apiurl'];
$url=$apiurl.'/v1/jobs/';
try {
$response = $client->post($url, ['json' => $arr]);
}
catch (\GuzzleHttp\Exception\BadResponseException $e) {
$req = $e->getRequest();
$resp =$e->getResponse();
echo "<pre>";
var_dump($req);
var_dump($resp);
die('ss');
}
try method-2
$arr=json_encode($arr);
$request = $client->post($url,array(
'content-type' => 'application/json'
),array());
$request->setBody($arr); #set body!
$response = $request->send();
then i am getting
InvalidArgumentException
No method is configured to handle the content-type config key
I'm just a beginner so I'm practicing basics and I know indentation is key and all that when writing in python but even though my indentation is correct (hopefully), it's reading it as if it's outside the loop. I know nested loops and crud make things all frizzy but I don't understand why when my indentation is fine.
f=open("output.txt","r+")
abc = ['a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y','z']
inp = raw_input("Enter a number between 0 and 27 inclusive: ")
for i in range(int(inp)):
print " "
c=0
s=""
while c<i and c<len(abc):
s+= abc[c]+" "
c+=1
c-=2
while c>=0:
s+= abc[c]+" "
c-=1
f.write(s)
print s
print "\nAll done"
f.close()
(Ps if some of the logic is out of whack sorry. All its supposed to do is go down the alphabet like a "a aba abcba abcdcba" and so on) Thank you
Ok so it works now but I'm trying to change it to an exec file but once I did, now when I double click to run it on the console it says: line 1: syntax error near unexpected token (' line 1: f = open("output.txt","w")' logout (ps I added a f.close() and fixed the logic) but when I run it regularly on the console (aka just "python filename.py" it runs just fine. Please help :)
I create one project and here is the folder structure:
mysite
mysite
__init__.py
settings.py
urls.py
wsgi.py
The manage.py is missing.
I am using Django version 1.4.20 and python version 2.6.6.
Is there anyone who has the same problem before?
Thank you.
I have tried all the existing resources but to no avail.
here is my app.yaml code:
version: 1
runtime: python27
api_version: 1
threadsafe: true
handlers:
- url: /.*
script: helloworld.app
Here is my webapp2 code:
import webapp2
class MainPage(webapp2.RequestHandler):
def get(self):
self.response.headers['Content-Type'] = 'text/plain'
self.response.write('Hello, World!')
app = webapp2.WSGIApplication([
('/', MainPage),
], debug=True)
All in my hello_world directory. I proceed to add existing application and point to the hello_world directory where helloworld.py (webapp2) script and the app.yaml code are living. I hit run on Google App Engine and it just stays loading, it never completes. Therefore, my http://localhost:8080/ page never loads.
I am taking Algorithms, Part I via Coursera and am looking to test the run time of the Quick Find, Quick Union, and Weighted Quick Union algorithms. The course is in Java, which I am unfamiliar with, so I have gone through and attempted to recreate the algorithms in Python, which I am more familiar with.
Now that I have everything implemented, I aim to test each function to verify run time/complexity. I had been thinking of using the timeit library, but that seems to be throwing incorrect results, e.g., Weighted Quick Union takes longer to complete than QuickUnion.
How can I verify that a Weighted Quick Union is in fact O(log n) and is faster than Quick Union? Here is what I have created and tried so far:
class QuickFind_Eager:
def __init__(self, nodes):
self.array = [num for num in range(nodes)]
# Joins two nodes into a component
def union(self, first_node, second_node):
for pos, val in enumerate(self.array):
if self.array[pos] == self.array[first_node]:
self.array[pos] = self.array[second_node]
# Checks if two nodes are in the same component
def connected(self, first_node, second_node):
return self.array[first_node] == self.array[second_node]
class QuickUnion_Lazy:
def __init__(self, nodes):
self.array = [num for num in range(nodes)]
# Follows parent pointers to actual root
def root(self, parent):
while parent != self.array[parent]:
parent = self.array[parent]
return parent
# Joins two nodes into a component
def union(self, first_node, second_node):
self.array[first_node] = self.array[second_node]
# Checks if two nodes are in the same component
def connected(self, first_node, second_node):
return self.root(first_node) == self.root(second_node)
class WeightedQuickUnion:
def __init__(self, nodes):
self.array = [num for num in range(nodes)]
self.weight = [num for num in range(nodes)]
# Follows parent pointers to actual root
def root(self, parent):
while parent != self.array[parent]:
parent = self.array[parent]
return parent
# Joins two nodes into a component
def union(self, first_node, second_node):
if self.root(first_node) == self.root(second_node):
return
if (self.weight[first_node] < self.weight[second_node]):
self.array[first_node] = self.root(second_node)
self.weight[second_node] += self.weight[first_node]
else:
self.array[second_node] = self.root(first_node)
self.weight[first_node] += self.weight[second_node]
# Checks if two nodes are in the same component
def connected(self, first_node, second_node):
return self.root(first_node) == self.root(second_node)
class WeightedQuickUnion_PathCompression:
def __init__(self, nodes):
self.array = [num for num in range(nodes)]
self.weight = [num for num in range(nodes)]
# Follows parent pointers to actual root
def root(self, parent):
while parent != self.array[parent]:
self.array[parent] = self.array[self.array[parent]]
parent = self.array[parent]
return parent
# Joins two nodes into a component
def union(self, first_node, second_node):
if self.root(first_node) == self.root(second_node):
return
if self.weight[first_node] < self.weight[second_node]:
self.array[first_node] = self.root(second_node)
self.weight[second_node] += self.weight[first_node]
else:
self.array[second_node] = self.root(first_node)
self.weight[first_node] += self.weight[second_node]
# Checks if two nodes are in the same component
def connected(self, first_node, second_node):
return self.root(first_node) == self.root(second_node)
import timeit
t = timeit.timeit(stmt="test_quickfind(QuickFind_Eager)", setup="from __main__ import QuickFind_Eager; from __main__ import test_quickfind", number=100000)
print(t)
t = timeit.timeit(stmt="test_quickfind(QuickUnion_Lazy)", setup="from __main__ import QuickUnion_Lazy; from __main__ import test_quickfind", number=100000)
print(t)
t = timeit.timeit(stmt="test_quickfind(WeightedQuickUnion)", setup="from __main__ import WeightedQuickUnion; from __main__ import test_quickfind", number=100000)
print(t)
t = timeit.timeit(stmt="test_quickfind(WeightedQuickUnion_PathCompression)", setup="from __main__ import WeightedQuickUnion_PathCompression; from __main__ import test_quickfind", number=100000)
print(t)
i got some code on python to open pptx files with LibreOffice
from pptx import Presentation
prs = Presentation('tes.pptx')
if i run this code on Raspbian, i got this error message
File "bukappt.pt", line 1, in <module>
from pptx import Presentation
File "/usr/local/lib/python2.7/dist-packages/pptx/_init_.py", line 15, in <module>
from pptx.api import Presentation # noqa
File "/user/local/lib/python2.7/dist-packages/pptx/api.py", line 14, in <module>
from ppx.package import Package
File "/usr/local/lib/python2.7/dist-packages/pptx/parts/image.py", line 13, in <module>
import Image as PIL_Image
ImportError: No modulenamed Image
how could i fix this?
I have an error when I insert data which form QlinEdit .
You can see the button1Clicked(self) method ,
I can't execute q.exec_("insert into COMPANY (id, name, age, address, salary) values (self.IDEdit.text(),self.NameEdit.text(),self.AgeEdit.text(),self.AddressEdit.text(),self.SalaryEdit.text())") this code .
But I can get the text when I use print(self.IDEdit.text()).
I don't understand why . I hope someone can help me.
#-*- coding: utf-8 -*-
import sys
import sqlite3
from PyQt4 import QtCore, QtGui
from PyQt4.QtSql import *
def createConnection():
db=QSqlDatabase.addDatabase("QSQLITE")
db.setDatabaseName("test.db")
db.open()
def createTable():
q=QSqlQuery() #
q.exec_("create table if not exists COMPANY (ID INT PRIMARY KEY NOT NULL,NAME TEXT NOT NULL,AGE INT NOT NULL,ADDRESS CHAR(50),SALARY REAL)")
q.exec_("commit")
class Model(QSqlTableModel):
def __init__(self,parent):
QSqlTableModel.__init__(self,parent)
self.setTable("COMPANY")
self.select()
self.setEditStrategy(QSqlTableModel.OnManualSubmit)
class TestWidget(QtGui.QWidget):
def __init__(self):
QtGui.QWidget.__init__(self)
vbox=QtGui.QVBoxLayout(self)
self.view=QtGui.QTableView()
self.model=Model(self.view)
self.view.setModel(self.model)
vbox.addWidget(self.view)
self.resize(900,300)
self.component()
def component(self):
ID = QtGui.QLabel('ID :', self)
ID.move(30, 180)
self.IDEdit = QtGui.QLineEdit(self)
self.IDEdit.move(70, 180)
self.IDEdit.setFocus()
Name = QtGui.QLabel('NAME :', self)
Name.move(180, 180)
self.NameEdit = QtGui.QLineEdit(self)
self.NameEdit.move(230, 180)
Age = QtGui.QLabel('AGE :', self)
Age.move(340, 180)
self.AgeEdit = QtGui.QLineEdit(self)
self.AgeEdit.move(380, 180)
Address = QtGui.QLabel('ADDRESS :', self)
Address.move(500, 180)
self.AddressEdit = QtGui.QLineEdit(self)
self.AddressEdit.move(560, 180)
Salary = QtGui.QLabel('SALARY :', self)
Salary.move(670, 180)
self.SalaryEdit = QtGui.QLineEdit(self)
self.SalaryEdit.move(730, 180)
button1= QtGui.QPushButton('submit', self)
button1.setGeometry(730, 240, 70, 30)
self.connect(button1, QtCore.SIGNAL('clicked()'),self.button1Clicked)
def button1Clicked(self):
q=QSqlQuery()
q.exec_("insert into COMPANY (id, name, age, address, salary) values (self.IDEdit.text(), self.NameEdit.text(),self.AgeEdit.text(),self.AddressEdit.text(),self.SalaryEdit.text())")
q.exec_("commit")
print (self.IDEdit.text())
print (self.NameEdit.text())
print (self.AgeEdit.text())
print (self.AddressEdit.text())
print (self.SalaryEdit.text())
if __name__=="__main__":
app=QtGui.QApplication(sys.argv)
createConnection()
createTable()
w=TestWidget()
w.show()
sys.exit(app.exec_())
I'm writing a script that is going to be used to fit some photoluminescence spectra with custom models and after creating a script using SciPy I learned that setting bounds on the fitting parameters is much easier if lmfit (http://ift.tt/1CbHc7t) is used instead so I decided to use that instead. Below is a script I wrote to fit one spectra with one Gaussian function (using 2 Gaussian functions is better, but I wanted to start with a simpler case).
from numpy import loadtxt, vstack, average, exp
import matplotlib.pyplot as plt
import lmfit
def sub_BG(data_array):
naujas_X = data_array[:,1]-average(data_array[1:10,1])
return vstack((data_array[:,0], naujas_X)).T
def gauss(x, a1, b1, c1):
return a1 * exp(-(x- b1)**2/ (2*c1**2))
#amp * exp(-(x-cen)**2 /wid)
data="single_spectra.txt"
spectra = loadtxt(open(data, 'r'), skiprows=2)
spectra_beBG = sub_BG(spectra)
plt.plot(spectra_beBG[:,0],spectra_beBG[:,1],'g')
mod1 = lmfit.Model(gauss)
pars = lmfit.Parameters()
# (Name, Value, Vary, Min, Max, Expr)
pars.add_many(('a1', 590, True, None, None, None),
('b1', 500, True, None, None, None),
('c1', 20, True, None, None , None))
out = mod1.fit(spectra_beBG[:,0], pars, x=spectra_beBG[:,1])
y = gauss(spectra_beBG[:,0],
out.best_values["a1"],
out.best_values["b1"],
out.best_values["c1"])
plt.plot(spectra_beBG[:,0], out.best_fit, "r--")
plt.plot(spectra_beBG[:,0], y, "b--")
print(out.fit_report())
This returns:
[[Model]]
Model(gauss)
[[Fit Statistics]]
# function evals = 77
# data points = 1024
# variables = 3
chi-square = 28469283.530
reduced chi-square = 27883.725
[[Variables]]
a1: 561.593868 +/- 8.255604 (1.47%) (init= 590)
b1: 100.129107 +/- 85.34384 (85.23%) (init= 500)
c1: 1.3254e+06 +/- 7.23e+05 (54.52%) (init= 20)
[[Correlations]] (unreported correlations are < 0.100)
C(b1, c1) = -0.892
C(a1, c1) = -0.763
C(a1, b1) = 0.685
If I change pars.add_many() to something closer to nature, for example:
pars.add_many(('a1', 590, True, 550, 630, None),
('b1', 500, True, 450, 650, None),
('c1', 30, True, 20, 70 , None))
I get this:
[[Model]]
Model(gauss)
[[Fit Statistics]]
# function evals = 147
# data points = 1024
# variables = 3
chi-square = 304708538.428
reduced chi-square = 298441.272
[[Variables]]
a1: 629.999937 +/- 0 (0.00%) (init= 590)
b1: 475.821359 +/- 0 (0.00%) (init= 500)
c1: 70 +/- 0 (0.00%) (init= 30)
[[Correlations]] (unreported correlations are < 0.100)
Help?
I have a uWSGI server with N amount of worker processes. Using the built in signal system I can effectively message the first available worker. I can also do something similar with the built in RPC system.
Does anyone here know of a way to send a signal to all workers?
Basically I would like to be able to notify the worker that it needs to reload a process local cache.
I am using the regex package to work with approximate regular expressions in Python. I do not understand the behavior I see when errors are counted in matches.
If I use the following regular expression with errors allowed:
>>> rex = '(HOUSE OF REPRESENTATIVES){e}'
I get exactly the behavior I expect, where a match is found with 20 errors. The 20 errors are due to the 20 lowercase letters:
>>> regex.match(rex, 'House of Representatives')
<regex.Match object; span=(0, 24), match='House of Representatives', fuzzy_counts=(20, 0, 0)>
When I attempt to something similar with added complexity, I get a result that makes no sense to me.
I am attempting to match strings that look like: "HOUSE OF REPRESENTATIVES\nWEDNESDAY, JANUARY 23, 1939\n".
>>> rex = '(HOUSE OF REPRESENTATIVES\n[A-Z]{3,6}DAY, [A-Z]{3,11} [0-9]{1,2}, [0-9]{4}\n){e}'
When I attempt to use this, I get an error number that I cannot understand.
>>> h1 = 'HOUSE OF REPRESENTATIVES\nWEDNESDAY, JANUARY 4, 1939\n'
>>> regex.match(rex, h1)
<regex.Match object; span=(0, 52), match='HOUSE OF REPRESENTATIVES\nWEDNESDAY, JANUARY 4, 1939\n', fuzzy_counts=(7, 0, 2)>
I get a match 9 fuzzy errors when I expect 0. What gives?
Note: When I use the BESTMATCH flag, I get 8 errors.
I have three Celery workers running on the same server, each running as a daemon with supervisord. I'm using RabbitMQ for the scheduling backend.
The Celery workers run tasks for a multi-tenant Django project. The supervisord conf file for each specifies an environment variable that instructions Django which settings file to load for the appropriate tenant. All three Django settings have a different BROKER_URL defined, specifying the proper BROKER_VHOST.
I'm testing by trying to queue an email-sending Celery task from each separate Django front-ends. I'm not receiving any explicit errors, but on each Django site, the task only seems to successfully execute 33% of the time. I can't find any proof in the /var/log/celery-* log files, but this seems suspiciously like each Celery worker is randomly grabbing a task not assigned to their tenant, seeing the VHOST doesn't match, and dropping it. Since there's only three of them, there's a 1 in 3 chance the task will be grabbed by the correct worker.
How do I fix this? Why is the Celery worker grabbing tasks from a different BROKER_VHOST?
I am trying to remove element from a list in Python.
AvailableLetters = ['a' 'b' 'c' 'd' 'e' 'f' 'g' 'h' 'i' 'j' 'k' 'l' 'm' 'n' 'o' 'p' 'q' 'r' 's' 't' 'u' 'v' 'w' 'x' 'y' 'z']
AvailableLetters.remove('a')
and I am getting the following error
AvailableLetters.remove('a')
ValueError: list.remove(x): x not in list
I'm looking at upgrading allauth on a live application from version 0.17 to 0.20.
It seems like the documentation doesn't have detailed instructions on how to do that. There used to be on up until version 0.17 in the Readme file, but not in later versions.
Anyone out there that's done it? @pennersr maybe? Or maybe I missed where that is documented?
For reference in case that matters, Python 2.7.3 and Django 1.6 (at this point).
full steps:
start django
start a celery worker
python manage.py celery worker --app=celery_worker:app -Ofair -n W1
upload a url list file, loop url list send each url to a task fetch_article
worker works
upload another url list file
worker no actions
views.py:
@csrf_exempt
def upload(request):
job_name = request.POST.get('job_name')
if not job_name:
return JsonResponse(JsonStatus.Error)
if len(request.FILES) == 1:
yq_data = request.FILES.values()[0]
else:
return JsonResponse(JsonStatus.Error)
job = Job.objects.create(name=job_name)
reader = csv.reader(yq_data, delimiter=',')
task_count = 0
next(reader)
for row in reader:
url = row[0].strip()
fetch_article.delay(job.id, url)
# fetch_article.apply_async(args=[job.id, url], queue=job.queue_name)
task_count += 1
# print 'qn%s' % job.queue_name
# rp = celery_app.control.add_consumer(queue=job.queue_name, reply=True)
# print rp
job.task_count = task_count
job.save()
return JsonResponse(JsonStatus.OK, msg=task_count)
tasks.py
@shared_task()
def fetch_article(job_id, url):
logger.info(u'fetch_article:%s' % url)
Processer = get_processor_cls(url)
a = Article(job_id=job_id, url=url)
try:
ap = Processer(url)
title, text = ap.process()
a.title = title
a.content = text
except Exception as e:
a.status = 2
a.error = e
logger.error(u'fetch_article:%s error:%s' % (url, e))
a.save()
To show a simple example, let's assume I have a simple function for z-score normalization:
def standardizing(array, columns, ddof=0):
ary_new = array.copy()
if len(ary_new.shape) == 1:
ary_new = ary_new[:, np.newaxis]
return (ary_new[:, columns] - ary_new[:, columns].mean(axis=0)) /\
ary_new[:, columns].std(axis=0, ddof=ddof)
And the results are what I expect:
>>> ary = np.array([[1, 10], [2, 9], [3, 8], [4, 7], [5, 6], [6, 5]])
>>> standardizing(ary, [0, 1])
array([[-1.46385011, 1.46385011],
[-0.87831007, 0.87831007],
[-0.29277002, 0.29277002],
[ 0.29277002, -0.29277002],
[ 0.87831007, -0.87831007],
[ 1.46385011, -1.46385011]])
However, let's say I don't want to return this array, but overwrite the values in-place. I am wondering why it doesn't work. For example,
def standardizing(array, columns, ddof=0):
ary_new = array.copy()
if len(ary_new.shape) == 1:
ary_new = ary_new[:, np.newaxis]
ary_new[:, columns] = (ary_new[:, columns] - ary_new[:, columns].mean(axis=0)) /\
ary_new[:, columns].std(axis=0, ddof=ddof)
# some more processing steps with ary_new
return ary_new
>>> ary = np.array([[1, 10], [2, 9], [3, 8], [4, 7], [5, 6], [6, 5]])
>>> standardizing(ary, [0, 1])
array([[-1, 1],
[ 0, 0],
[ 0, 0],
[ 0, 0],
[ 0, 0],
[ 1, -1]])
I have a file A with sequence ID and also the information of binding site location. I would like to extract the location information only without the A,T,C,G information. The shorter sequence above longer sequence shows its location and every number on the left for example in File A the value 451 is the location value on the left. I would like to get the location of short sequence on longer sequence which is start with 453 (start site) and obtain the length of shorter sequence which is size 21 and add to 453 to get the end site which is 474. Can anyone help me?
File A.txt
chr1:152806601-152807450
TTCAGCACCATGGACAGCGCC
451 GGCTTCAGCACCACGGACAGCGCCCCACCCGCGGCCCTCCCCCCGGCGGCGCGCTCCAGCCGGTGTAGGCGAGGC
TTCAGCACCATGGACAGCGCC
751 AGAGCCCCCCGGGACTGCAGAGAGCACCTGGGAGGCTGGACTGGGAACGAGACATACTCGAAGGAGTAAGTGAAG
chr10:125364276-125364825
TTCAGCACCATGGACAGCGCC
301 CAGTAATGTGGGGTTGTGGTCAGCACCATGGACAGCTCCCCTGTTGCTTCATATTGAGGAATAGGAAAGCGCCGC
TTCAGCACCATGGACAGCGCC
376 TATCTCCGGATCCTGGCTAGCTCCAGCCACTGCAGGTAACTGTCTTGAATGGGCTTAGAAACATGGTGATGTCTG
Desired output
chr1:152806601-152807450 453 474
chr1:152806601-152807450 757 778
chr10:125364276-125364825 318 339
chr10:125364276-125364825 378 399
Example code
import re
with open("A.txt", "r") as f:
lines = f.readlines()
label_ptrn = re.compile("") # insert regular expression sequence ID
line_ptrn = re.compile("") # insert regular expression start site
inner_ptrn = re.compile("") # insert regular expression end site
all_matches = []
for line in lines:
m = label_ptrn.match(line)
if m:
label = m.groupdict().get("label")
continue
m = line_ptrn.match(line)
if m:
start = m.groupdict().get("start_value")
sequence = m.groupdict().get("sequence")
mi = inner_ptrn.search(sequence)
if not mi:
continue
span = mi.span()
all_matches.append((label, int(start)+span[0], int(start)+span[1]))
with open("A_ouput.bed", "w+b") as f:
for m in all_matches:
f.write('%s\t%i\t%i\n' % m)
I am using a Python ScriptEngine in my Java application to execute a Python script that imports various other Python scripts. Once the main script successfully completes and the eval method returns, I set the engine object reference to null and call the garbage collector. Next I go and edit one of the scripts that was imported by the main script and save it. Next I run the method to execute the main Python script as before by creating a new ScriptEngine object and calling eval, but when the main script runs it does not pick up the changes to the imported script that I made. Obviously the imported scripts are being cached somewhere by something, maybe Jython? I do not want to call reload in the Python script. There must be a way to tell whatever is doing the caching to flush or clear itself. Anyone found a solution to this problem? I am using NetBeans 8.0.2, Java 1.8 update 45, and Jython 2.7 on Windows 7.
I use the svm model in scikit-learner to predict handwriting in MNIST.
However, the result I got is confused. When I use trained model to predict training set that already used in learning procedure the accuracy is 100%
When dealing with the test data I only got about 11% accuracy.
I cannot find out the reason except overfitting. Does overfitting has such a strong influence on the outcome?
# coding:utf-8
from numpy import *
from sklearn import svm
from sklearn.externals import joblib
def loadData(fileName):
fr = open(fileName)
numFeat = len(fr.readline().split(',')) - 1
featMatTrain = []
labelVecTrain = []
featMatTest = []
labelVecTest = []
i = 0
for line in fr.readlines():
i = i + 1
if i != 1 and i <=30000:
curLine = line.strip().split(',')
curLine = map(float,curLine)
labelVecTrain.append(curLine[0])
featMatTrain.append(curLine[1:numFeat])
if i >= 30000:
curLine = line.strip().split(',')
curLine = map(float,curLine)
labelVecTest.append(curLine[0])
featMatTest.append(curLine[1:numFeat])
print '*************************** the training data we got: *****************************'
print 'featMat:''type of element:',type(featMatTrain) ,'shape of featMat:', shape(featMatTrain)
print 'labelVec:''type of element:',type(labelVecTrain),'shape of labelVec:',shape(labelVecTrain)
print 'featMat:''type of element:',type(featMatTest) ,'shape of featMat:', shape(featMatTest)
print 'labelVec:''type of element:',type(labelVecTest),'shape of labelVec:',shape(labelVecTest)
return array(featMatTrain),array(labelVecTrain),array(featMatTest),array(labelVecTest)
featMatTrain,labelVecTrain,featMatTest,labelVecTest= loadData('C:/Users/sun/Desktop/train.csv')
clf = svm.SVC()
clf.fit(featMatTrain,labelVecTrain)
joblib.dump(clf,'svmModel.pkl')
print '***************** we finish training **********************'
labelVecPredict1 = clf.predict(featMatTrain)
labelVecPredict2 = clf.predict(featMatTest)
print '***************** we finish predicting **********************'
count1 = 0.0
for i in range(len(featMatTrain)):
if labelVecPredict1[i] == labelVecTrain[i]:
count1 = count1 + 1
print '************* the result of predicting training set ***************'
print 'the number of figures that predict right: ',count1
print 'the accuary is :',count1/len(featMatTrain)
count2 = 0.0
for i in range(len(featMatTest)):
if labelVecPredict2[i] == labelVecTest[i]:
count2 = count2 + 1
print '************ the result to predicting testing set ************'
print 'the number of figures that predict right:',count2
print 'the accuary is:',count2/len(featMatTest)
I am trying to update a 3D plot using matplotlib. I am collecting data using ROS. I want to update the plot as I get data. I have looked around and found this, Dynamically updating plot in matplotlib
but I cannot get it to work. I am very new to python and do not full understand how it works yet. I apologize if my code is disgusting.
I keep get this error.
[ERROR] [WallTime: 1435801577.604410] bad callback: <function usbl_move at 0x7f1e45c4c5f0>
Traceback (most recent call last):
File "/opt/ros/indigo/lib/python2.7/dist-packages/rospy/topics.py", line 709, in _invoke_callback
cb(msg, cb_args)
File "/home/nathaniel/simulation/src/move_videoray/src/move_filtered.py", line 63, in usbl_move
if filter(pos.pose.position.x,pos.pose.position.y,current.position.z):
File "/home/nathaniel/simulation/src/move_videoray/src/move_filtered.py", line 127, in filter
plt.draw()
File "/usr/lib/pymodules/python2.7/matplotlib/pyplot.py", line 555, in draw
get_current_fig_manager().canvas.draw()
File "/usr/lib/pymodules/python2.7/matplotlib/backends/backend_tkagg.py", line 349, in draw
tkagg.blit(self._tkphoto, self.renderer._renderer, colormode=2)
File "/usr/lib/pymodules/python2.7/matplotlib/backends/tkagg.py", line 13, in blit
tk.call("PyAggImagePhoto", photoimage, id(aggimage), colormode, id(bbox_array))
RuntimeError: main thread is not in main loop
This is the code I am trying to run
#!/usr/bin/env python
'''
Ths program moves the videoray model in rviz using
data from the /usble_pose node
based on "Using urdf with robot_state_publisher" tutorial
'''
import rospy
import roslib
import math
import tf
#import outlier_filter
from geometry_msgs.msg import Twist, Vector3, Pose, PoseStamped, TransformStamped
from matplotlib import matplotlib_fname
from mpl_toolkits.mplot3d import Axes3D
import sys
from matplotlib.pyplot import plot
from numpy import mean, std
import matplotlib as mpl
import numpy as np
import pandas as pd
import random
import matplotlib.pyplot as plt
#plt.ion()
import matplotlib
mpl.rc("savefig", dpi=150)
import matplotlib.animation as animation
import time
#filter stuff
#window size
n = 10
#make some starting values
#random distance
md =[random.random() for _ in range(0, n)]
#random points
x_list = [random.random() for _ in range(0, n)]
y_list =[random.random() for _ in range(0, n)]
#set up graph
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
#ax.scatter(filt_x,filt_y,filt_depth,color='b')
#ax.scatter(outlier_x,outlier_y,outlier_depth,color='r')
ax.set_xlabel('X')
ax.set_ylabel('Y')
ax.set_zlabel('Z')
ax.set_title('XY Outlier rejection \n with Mahalanobis distance and rolling mean3')
#set the robot at the center
#//move the videoray using the data from the /pose_only node
def usbl_move(pos,current):
broadcaster = tf.TransformBroadcaster()
if filter(pos.pose.position.x,pos.pose.position.y,current.position.z):
current.position.x = pos.pose.position.x
current.position.y = pos.pose.position.y
broadcaster.sendTransform( (current.position.x,current.position.y,current.position.z),
(current.orientation.x,current.orientation.y,current.orientation.z,current.orientation.w),
rospy.Time.now(), "odom", "body" )
#move the videoray using the data from the /pose_only node
def pose_move(pos,current):
#pos.position.z is in kPa, has to be convereted to depth
# P = P0 + pgz ----> pos.position.z = P0 + pg*z_real
z_real = -1*(pos.position.z -101.325)/9.81;
#update the movement
broadcaster = tf.TransformBroadcaster()
current.orientation.x = pos.orientation.x
current.orientation.y = pos.orientation.y
current.orientation.z = pos.orientation.z
current.orientation.w = pos.orientation.w
current.position.z = z_real
broadcaster.sendTransform( (current.position.x,current.position.y,current.position.z),
(current.orientation.x,current.orientation.y,current.orientation.z,current.orientation.w),
rospy.Time.now(), "odom", "body" )
#call the fitle the date
def filter(x,y,z):
# update the window
is_good = False
x_list.append(x)
y_list.append(y)
x_list.pop(0)
y_list.pop(0)
#get the covariance matrix
v = np.linalg.inv(np.cov(x_list,y_list,rowvar=0))
#get the mean vector
r_mean = mean(x_list), mean(y_list)
#subtract the mean vector from the point
x_diff = np.array([i - r_mean[0] for i in x_list])
y_diff = np.array([i - r_mean[1] for i in y_list])
#combinded and transpose the x,y diff matrix
diff_xy = np.transpose([x_diff, y_diff])
# calculate the Mahalanobis distance
dis = np.sqrt(np.dot(np.dot(np.transpose(diff_xy[n-1]),v),diff_xy[n-1]))
# update the window
md.append( dis)
md.pop(0)
#find mean and standard standard deviation of the standard deviation list
mu = np.mean(md)
sigma = np.std(md)
#threshold to find if a outlier
if dis < mu + 1.5*sigma:
#filt_x.append(x)
#filt_y.append(y)
#filt_depth.append(z)
ax.scatter(x,y,z,color='b')
is_good = True
else:
ax.scatter(x,y,z,color='r')
plt.draw()
return is_good
if __name__ == '__main__':
#set up the node
rospy.init_node('move_unfiltered', anonymous=True)
#make a broadcaster foir the tf frame
broadcaster = tf.TransformBroadcaster()
#make intilial values
current = Pose()
current.position.x = 0
current.position.y = 0
current.position.z = 0
current.orientation.x = 0
current.orientation.y = 0
current.orientation.z = 0
current.orientation.w = 0
#send the tf frame
broadcaster.sendTransform( (current.position.x,current.position.y,current.position.z),
(current.orientation.x,current.orientation.y,current.orientation.z,current.orientation.w),
rospy.Time.now(), "odom", "body" )
#listen for information
rospy.Subscriber("/usbl_pose", PoseStamped, usbl_move,current)
rospy.Subscriber("/pose_only", Pose, pose_move, current);
rospy.spin()
I am new to python. I have query regarding un-hardcoding object names(if condition) in python script. I have fruit = [ Apple, Mango, Pineapple, Banana, Oranges] and size = [ small, medium , big] Currently I write code as below:
if (fruit == apple, size == small):
statement 1
statement 2
elif (fruit == apple, size == medium):
statement 1
statement 2
elif (fruit == apple, size == big):
statement 1
statement 2
elif (fruit == Mango, size == small):
statement 1
statement 2
elif (fruit == Mango, size = medium):
statement 1
statement 2
How can I avoid writing multiple if...else conditions?
The default django radio button renderer outputs the <input> within the <label> tag & I've been asked to output them at the same level in the DOM so have written a custom renderer.
The trouble is, the field isn't rendering with safe HTML so in the browser I'm just seeing the field label, then the raw html for the choices.
My form field is declared as so;
class SignupForm(forms.ModelForm):
my_field = forms.NullBooleanField(
widget=forms.widgets.RadioSelect(
choices=FIELD_CHOICES,
renderer=MyRadioFieldRenderer
),
required=True,
initial=True
)
And my custom renderer;
class MyChoiceInput(forms.widgets.ChoiceInput):
def render(self, name=None, value=None, attrs=None, choices=()):
if self.id_for_label:
label_for = format_html(' for="{0}"', self.id_for_label)
else:
label_for = ''
return format_html(
'{0}<label{1}>{2}</label>', self.tag(), label_for, self.choice_label
)
class RadioChoiceInput(MyChoiceInput):
input_type = 'radio'
def __init__(self, *args, **kwargs):
super(RadioChoiceInput, self).__init__(*args, **kwargs)
self.value = force_text(self.value)
class MyRadioFieldRenderer(forms.widgets.ChoiceFieldRenderer):
choice_input_class = RadioChoiceInput
I've looked over the ChoiceFieldRenderer which I'm inheriting & it's render function returns a mark_safe() string, so why does my field just render raw HTML?
consider a list a = ['Anatomy', 'of', 'the', 'Human', 'Body', 'Henry', 'Gray', 'CONTENTS', 'I.', 'Embryology', '1.', 'The', 'Animal', 'Cell', '2.', 'The', 'Ovum', '3.', 'The', 'Spermatozoön', '4.', 'Fertilization', 'of', 'the', 'Ovum', '5.', 'Segmentation', 'of', 'the', 'Fertilized', 'Ovum', '6.', 'The', 'Neural', 'Groove', 'and', 'Tube'] How would you create a program that picked out elements based on other elements easily? Say I want the elements between 'The' and int + '.' to be stored in a list. Or store 'Ovum' in a list named 'Fertilization' because 'Ovum' is between 'The' and int + '.' and 'Fertilization' is before 'of' and after int + '.'
Newby here. I am having trouble with manipulating my grid after i use the meshgrid. The code is as follows:
def XV(v):
return v**2
"Create the points over v and u"
v = linspace(-10,10,20)
u = np.random.normal(0,10,20)
"create the grid"
V, U = meshgrid(v,u)
"Create the order flow Y"
Y = XV(V) + U
"Create joint probability function"
def FVY(Y,XV,V):
return (1/2*pi)*exp(-((Y-XV(V))/2))*exp(-(V**2)/2)
There is no error in the creation of Y, but when I go to create FVY I get the following error.
TypeError: only length-1 arrays can be converted to Python scalars
As far as I can see there is nothing different in the construction of these two variables and so I can't understand the error.
Thanks for any help.
I am trying to insert data from a dataframe df into a numpy array matrix_of_coupons_and_facevalues. However, I basically need to add the value associated with a row of df['Coupon'] into each column of a corresponding row of the array for as many columns as the number numberofcoupon_payments = row1['months_to_maturity']/6. I get the error ValueError: could not broadcast input array from shape (1,2) into shape (1,61) in the line np.insert(matrix_of_coupons_and_facevalues, row_no+1, rowtobeadded, 0) and I understand why, but I don't know how to proceed.
The code I am using is as follows:
matrix_of_coupons_and_facevalues = np.zeros((number_of_rows_and_columns, number_of_rows_and_columns))
rowtobeadded = np.zeros(number_of_rows_and_columns)
for (i1,row1) in df.iterrows():
numberofcoupon_payments = row1['months_to_maturity']/6
for row_no in range(int(number_of_rows_and_columns)):
for index_no in range(int(numberofcoupon_payments)):
coupon = row1['coupon']
rowtobeadded = np.full((1, numberofcoupon_payments), coupon)
np.insert(matrix_of_coupons_and_facevalues, row_no+1, rowtobeadded, 0)
Edit: The dataframe df looks like this:
months_to_maturity on_the_run_dt asset_id \
0 5 2015-07-02 00:00:00.0 00102CC07F4B02CA
1 6 2015-06-25 00:00:00.0 00102CD0FB2A023F
2 11 2015-04-02 00:00:00.0 00102CFED3C500D4
3 12 2015-06-25 00:00:00.0 00102C37122B0230
4 23 2015-03-02 00:00:00.0 00102C76082B0069
orig_iss_dt maturity_dt pay_freq_cd coupon \
0 2015-07-02 00:00:00.0 2015-12-31 00:00:00.0 NaN 0.000
1 2015-06-25 00:00:00.0 2015-12-24 00:00:00.0 NaN 0.000
2 2015-04-02 00:00:00.0 2016-03-31 00:00:00.0 NaN 0.000
3 2015-06-25 00:00:00.0 2016-06-23 00:00:00.0 NaN 0.000
4 2015-03-02 00:00:00.0 2017-02-28 00:00:00.0 2 0.500
closing_price cpn_type_cd months_to_maturity_1 FACE_VALUE
0 99.944389 FXDI 5 24000101.6
1 99.960889 FXDI 6 24000366.4
2 99.866806 FXDI 11 25000267.5
Desired output Array: For example I need the array to look like this, if the months_to_maturity column of df has values 6,12,18:
array([[coupon 0 0 0],
[coupon coupon 0 0],
[coupon coupon coupon 0]])
Thank You
I am learning Python and am trying to scrape this page (http://ift.tt/1H1uQNP) for a specific value on the dropdown menu. After that I need to click each item on the resulted table to retrieve the specific information. I am able to select the item and retrieve the information on the webdriver. But I do not know how to pass the response url to the crawlspider.
driver = webdriver.Firefox()
driver.get('http://ift.tt/1H1uQNP')
more_btn = WebDriverWait(driver, 20).until(
EC.visibility_of_element_located((By.ID, '_button_select'))
)
more_btn.click()
## select specific value from the dropdown
driver.find_element_by_css_selector("select#tabJcwyxt_jiebie > option[value='teyaoxgrs']").click()
driver.find_element_by_css_selector("select#tabJcwyxt_jieci > option[value='d11jie']").click()
search2 = driver.find_element_by_class_name('input_a2')
search2.click()
time.sleep(5)
## convert html to "nice format"
text_html=driver.page_source.encode('utf-8')
html_str=str(text_html)
## this is a hack that initiates a "TextResponse" object (taken from the Scrapy module)
resp_for_scrapy=TextResponse('none',200,{},html_str,[],None)
## convert html to "nice format"
text_html=driver.page_source.encode('utf-8')
html_str=str(text_html)
resp_for_scrapy=TextResponse('none',200,{},html_str,[],None)
So this is where I am stuck. I was able to query using the above code. But How can I pass resp_for_scrapy to the crawlspider? I put resp_for_scrapy in place of item but that didn't work.
## spider
class ProfileSpider(CrawlSpider):
name = 'pccprofile2'
allowed_domains = ['cppcc.gov.cn']
start_urls = ['http://ift.tt/1H1uQNP']
def parse(self, resp_for_scrapy):
hxs = HtmlXPathSelector(resp_for_scrapy)
for post in resp_for_scrapy.xpath('//div[@class="table"]//ul//li'):
items = []
item = Ppcprofile2Item()
item ["name"] = hxs.select("//h1/text()").extract()
item ["title"] = hxs.select("//div[@id='contentbody']//tr//td//text()").extract()
items.append(item)
##click next page
while True:
next = self.driver.findElement(By.linkText("下一页"))
try:
next.click()
except:
break
return(items)
Any suggestions would be greatly appreciated!!!!
I installed Python 3.4.0 64 bit and gdal file release-1400-x64-gdal-1-11-1-mapserver-6-4-1.zip from http://ift.tt/1EbvBCh. I found the binding from http://ift.tt/1jTmHBw and the filename is GDAL-1.11.2-cp34-none-win_amd64. I successfully install these files and import gdal. However, when I run the following command within the Python IDE to merge files 1 2 and 3, I got an error
>>> gdal_merge.py -o out.tif 1.tif 2.tif 3.tif
File "<console>", line 1
gdal_merge.py -o out.tif 1.tif 2.tif 3.tif
^
SyntaxError: invalid syntax
I specifically check to see if I can import gdal_merge as below
>>> import gdal_merge
and it was ok. I appreciate if anybody could help with this issue.
this is probably a simple problem so hopefuly its easy for someone to point out my mistake or if this is even possible.
I have an object that has multiple objects as properties. I want to be able to dynamically set the properties of these objects like so:
class Person(object):
def __init__(self):
self.pet = Pet()
self.residence = Residence()
class Pet(object):
def __init__(self,name='Fido',species='Dog'):
self.name = name
self.species = species
class Residence(object):
def __init__(self,type='House',sqft=None):
self.type = type
self.sqft=sqft
if __name__=='__main__':
p=Person()
setattr(p,'pet.name','Sparky')
setattr(p,'residence.type','Apartment')
print p.__dict__
The output is:
{'pet': <main.Pet object at 0x10c5ec050>, 'residence': <main.Residence object at 0x10c5ec0d0>, 'pet.name': 'Sparky', 'residence.type': 'Apartment'}
As you can see, rather then having the name attribute set on the pet object of the person, a new attribute "pet.name" is created.
I cannot specify person.pet to setattr because different child-objects will be set by the same method, which is parsing some text and filling in the object attributes if/when a relevant key is found.
Is there a easy/built in way to accomplish this?
Or perhaps I need to write a recursive function to parse the string and call getattr multiple times until the necessary child-object is found and then call setattr on that found object?
Thank you!
My business user likes to use the then sentence "It should be created", where it is determined by the context of the scenario. For example:
Given I have gift certificate for "<name>"
When I enter the gift certificate
Then It should be created
or
Given Customer order return for order "<order_no>"
When I create the customer order return
Then It should be created
In the "Then It should be created", I would like to retrieve either the created gift certificate or customer order return for comparison. However, they have completely different API and object.
Firstly, is there a way to do this in Python Behave without getting "Exception AmbiguousStep:"?
If not, what would be best practice in BDD world for this without forcing user to have to repeat themselves constantly by saying "Then The gift certificate should be created" or "Then The customer order return should be created"?
Thanks.
I am trying to run a Linux server configuration for a catalog project. I am using: Apache2, Flask, and SQLAlchemy and I had to install and configure PostgreSQL. The site I am trying to run is the Public IP Address: http://52.27.140.219/. I am happy to provide greater detail. I am new with Linux and many things discussed here, but I believe my work is strong, except for the end problem below.
Important steps in Terminal:
From: (venv)grader@ip-10-20-8-44:/var/www/catalog/catalog$ sudo service apache2 restart [sudo] password for grader: * Restarting web server apache2 [ OK ]
Error Logs
(venv)grader@ip-10-20-8-44:/var/www/catalog/catalog$ sudo tail -20 /var/log/apache2/error.log
[Wed Jul 01 16:51:36.873041 2015] [:error] [pid 14600:tid 140294116747008] [client 73.221.39.5:60704] Traceback (most recent call last):
[Wed Jul 01 16:51:36.873059 2015] [:error] [pid 14600:tid 140294116747008] [client 73.221.39.5:60704] File "/var/www/catalog/catalog.wsgi", line 7, in <module>
[Wed Jul 01 16:51:36.873105 2015] [:error] [pid 14600:tid 140294116747008] [client 73.221.39.5:60704] from catalog import app as application
[Wed Jul 01 16:51:36.873117 2015] [:error] [pid 14600:tid 140294116747008] [client 73.221.39.5:60704] File "/var/www/catalog/catalog/__init__.py", line 35, in <module>
[Wed Jul 01 16:51:36.873257 2015] [:error] [pid 14600:tid 140294116747008] [client 73.221.39.5:60704] open('client_secrets.json', 'r').read())['web']['client_id']
[Wed Jul 01 16:51:36.873279 2015] [:error] [pid 14600:tid 140294116747008] [client 73.221.39.5:60704] IOError: [Errno 2] No such file or directory: 'client_secrets.json'
[Wed Jul 01 16:53:36.405496 2015] [mpm_event:notice] [pid 14596:tid 140294245513088] AH00491: caught SIGTERM, shutting down
[Wed Jul 01 16:53:37.387879 2015] [mpm_event:notice] [pid 14715:tid 140705447798656] AH00489: Apache/2.4.7 (Ubuntu) mod_wsgi/3.4 Python/2.7.6 configured -- resuming normal operations
[Wed Jul 01 16:53:37.387939 2015] [core:notice] [pid 14715:tid 140705447798656] AH00094: Command line: '/usr/sbin/apache2'
[Wed Jul 01 17:35:23.312436 2015] [mpm_event:notice] [pid 14715:tid 140705447798656] AH00491: caught SIGTERM, shutting down
[Wed Jul 01 17:35:24.360945 2015] [mpm_event:notice] [pid 15120:tid 140719357745024] AH00489: Apache/2.4.7 (Ubuntu) mod_wsgi/3.4 Python/2.7.6 configured -- resuming normal operations
[Wed Jul 01 17:35:24.361007 2015] [core:notice] [pid 15120:tid 140719357745024] AH00094: Command line: '/usr/sbin/apache2'
[Wed Jul 01 17:35:35.741239 2015] [:error] [pid 15123:tid 140719262549760] [client 73.221.39.5:61118] mod_wsgi (pid=15123): Target WSGI script '/var/www/catalog/catalog.wsgi' cannot be loaded as Python module.
[Wed Jul 01 17:35:35.741269 2015] [:error] [pid 15123:tid 140719262549760] [client 73.221.39.5:61118] mod_wsgi (pid=15123): Exception occurred processing WSGI script '/var/www/catalog/catalog.wsgi'.
[Wed Jul 01 17:35:35.741302 2015] [:error] [pid 15123:tid 140719262549760] [client 73.221.39.5:61118] Traceback (most recent call last):
[Wed Jul 01 17:35:35.741317 2015] [:error] [pid 15123:tid 140719262549760] [client 73.221.39.5:61118] File "/var/www/catalog/catalog.wsgi", line 7, in <module>
[Wed Jul 01 17:35:35.741359 2015] [:error] [pid 15123:tid 140719262549760] [client 73.221.39.5:61118] from catalog import app as application
[Wed Jul 01 17:35:35.741369 2015] [:error] [pid 15123:tid 140719262549760] [client 73.221.39.5:61118] File "/var/www/catalog/catalog/__init__.py", line 35, in <module>
[Wed Jul 01 17:35:35.741483 2015] [:error] [pid 15123:tid 140719262549760] [client 73.221.39.5:61118] open('client_secrets.json', 'r').read())['web']['client_id']
[Wed Jul 01 17:35:35.741501 2015] [:error] [pid 15123:tid 140719262549760] [client 73.221.39.5:61118] IOError: [Errno 2] No such file or directory: 'client_secrets.json'
When I sudo nano /etc/apache2/sites-available/catalog.conf, I have:
WSGIPythonPath /var/www/catalog/catalog/venv/:/var/www/catalog/catalog/v$
<VirtualHost *:80>
ServerName 52.27.140.219
ServerAdmin admin@52.27.140.219
ServerAlias c-73-221-39-5.hsd1.wa.comcast.net
WSGIScriptAlias / /var/www/catalog/catalog.wsgi
<Directory /var/www/catalog/catalog/>
Order allow,deny
Allow from all
</Directory>
Alias /static /var/www/catalog/catalog/static
<Directory /var/www/catalog/catalog/static/>
Order allow,deny
Allow from all
</Directory>
ErrorLog ${APACHE_LOG_DIR}/error.log
LogLevel warn
CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>
I'm struggling to get my new UI window parented to the main Maya window. I've been following a few tutorials, but not having much luck combining them.
With the below code (and you can use any UI file you have) I get 2 windows: one that is blank, but parented to the Maya window, and one that is the proper UI file, but is not parented to the Maya window.
I understand that there are 2 times that ui.show is being called, however I don't really understand why the UI's are different.
My end goal is to get the version with the Class & UI file parented to the main Maya window.
import sys
from PySide.QtUiTools import *
from PySide.QtCore import *
from PySide.QtGui import *
def getMayaWindow():
pointer = mui.MQtUtil.mainWindow()
QtGui.QMainWindow.styleSheet(shiboken.wrapInstance(long(pointer), QtGui.QWidget))
return shiboken.wrapInstance(long(pointer), QtGui.QWidget)
class TestApp(QMainWindow):
def __init__(self, parent = getMayaWindow()):
super(TestApp, self).__init__(parent)
loader = QUiLoader()
self.ui = loader.load('c:/pyqt_tutorial.ui')
self.ui.show()
if __name__ == "__main__":
try:
ui.close()
except:
pass
ui = TestApp()
ui.show()
I changed this list
orig_list=['"jason","hello1,hello2,hello3","somegroup2","bundle1","loc1"', '"ruby","hello","somegroup","bundle2","loc2"', '"sam","hello3,hello2","somegroup3,somegroup4","bundle2","loc3"']
new_list=[x.split(",") for x in orig_list]
new_list=[['"jason"', '"hello1', 'hello2', 'hello3"', '"somegroup2"', '"bundle1"', '"loc1"'], ['"ruby"', '"hello"', '"somegroup"', '"bundle2"', '"loc2"'], ['"sam"', '"hello3', 'hello2"', '"somegroup3', 'somegroup4"', '"bundle2"', '"loc3"']]
what my intent is to get
[['jason', 'hello1,hello2,hello3', 'somegroup2', 'bundle1', 'loc1'], ['ruby', 'hello', 'somegroup', 'bundle2', 'loc2'], ['sam', 'hello3,hello2', 'somegroup3,somegroup4', 'bundle2', 'loc3']]
Is it possible to do it inplace and not creating a new one?
Update : I can have some elements in double quotes, all in double quotes, no double quotes and same in single quotes.
I have a requirement to execute multiple Python statements and few of them might fail during execution, even after failing I want the rest of them to be executed.
Currently, I am doing:
try:
wx.StaticBox.Destroy()
wx.CheckBox.Disable()
wx.RadioButton.Enable()
except:
pass
If any one of the statements fails, except will get executed and program exits. But what I need is even though it is failed it should run all three statements.
How can I do this in Python?
I'd like to generate random numbers that follow a dropping linear frequency distribution, take n=1-x for an example.
The numpy library however seems to offer only more complex distributions.
(ALERT: Conceptual, expert-opinion-seeking question)
I have an Excel file - my profit spreadsheet - which has one tab per product range. The tabs are defined and named, but apart from that, the file contains no data (yet). I wish to use openpyxl to:
When all is said and done, we end up with a value for profit, per product, per day.
The resultant profit spreadsheet should look something like this (the Cars tab is shown, but all tabs will have an identical structure):
Since the cell values in Column A of each tab will be product names and will likely remain constant, the number of rows will also likely remain constant, per tab, day on day. As such, the product names may lend themselves to the keys of dicts, per product range (tab).
If I set up some functions thus:
def parseExpenseSpreadsheet(productLineExpensesFilename):
# some standard openpyxl code to extract the values in the Revenue column
# of the sales spreadsheet
return productRangesExpensesDict # dict of product&expenses pairs
def parseSalesSpreadsheet(salesFilename):
#similar to parseExpenseSpreadsheet() but returns productRangesSalesDict dict
def calculateProfitPerProductRange(productRangesExpensesDict, productRangesSalesDict)
return y # dict of (sales minus profit per product) pairs
def appendToProfitSpreadsheet(profitSpreadsheet, expensesPerProductRange)
addNewDayData(expensesPerProductRange)
def addNewDayData(expensesPerProductRange):
for productExpensePair in expensesPerProductRange:
profit[productExpensePair] = calculateProfitPerProductRange()[productExpensePair]
# some standard openpyxl code to write the resultant
# profit dict. appendToProfitSpreadsheet() will determine which tab to write the
# dict to.
and roll it out as follows:
def main():
# some standard openpyxl code to open the profit spreadsheet for
# appending
tabs = ['Cars', 'Bikes', 'Vans']
productRangeExpensesDictOfDicts = []
for tab in tabs:
expensesPerProductRange = parseExpenseSpreadsheet(tab)
appendToProfitSpreadsheet(profitSpreadsheet, expensesPerProductRange)
some of my design just doesn't smell right e.g.
profit[product] = calculateProfitPerProductRange()[product]And in general it all feels a tad convoluted. Before I go test it, I welcome thoughts on how it can be improved (and indeed, if there are any glaring bugs).
Have a PySpark broadcast value with content like this:
[('b000jz4hqo', {'rom': 2.4051362683438153, 'clickart': 56.65432098765432, '950': 254.94444444444443, 'image': 3.6948470209339774, 'premier': 9.27070707070707, '000': 6.218157181571815, 'dvd': 1.287598204264871, 'broderbund': 22.169082125603865, 'pack': 2.98180636777128}), ('b0006zf55o', {'laptops': 11.588383838383837, 'desktops': 12.74722222222222, 'backup': 2.8015873015873014, 'win': 0.501859142607174, 'ca': 9.10515873015873, 'v11': 50.98888888888888, '30u': 84.98148148148148, '30pk': 254.94444444444443, 'desktop': 2.23635477582846, '1': 0.3231235037318687, 'arcserve': 24.28042328042328, 'computer': 0.6965695203400122, 'lap': 127.47222222222221, 'oem': 46.35353535353535, 'international': 9.44238683127572, 'associates': 7.284126984126985})]
So it is a key->list broadcast variable.
Attempts to convert broadcast.value into a dictionary results in
TypeError: unhashable type: 'dict'
Using code like
from itertools import izip
amazonWeightsBroadcast = sc.broadcast(amazonWeightsRDD.collect())
i = iter(amazonWeightsBroadcast.value)
amazonWeightsDict = dict(izip(i, i))
So if it's not possible to convert a broadcast variable into a dictionary, what will be a better solution to lookup a value-list by a key?
Python 2.7.6 Spark 1.3.1
I have one file, let's call it foo.py. It does a couple of things, including sending some data over a serial port and emailing the response that comes back.
I have another file, which looks something like this:
iteration = 0
while True:
iteration += 1
// do some stuff here every time
if iteration%5 == 0:
import foo
time.sleep (100)
I'm aware there are some broader problems here with the elegance (or lack thereof) of an independent counter, but putting that aside - the serial transmission / email only works the first time it's triggered. Subsequent loops at a multiple of 5 (which will trigger the modulo 5 == 0) do nothing.
Does my imported version of foo.py get cached, and avoid triggering on subsequent runs? If yes, how else can I call that code repeatedly from within my looping script? Should I just include it inline?
Thanks for any tips!
I have recently decided to start doing some web application development for some side project ideas that I have. I am trying to use Flask to begin doing this, and I am going through some tutorials to get the hang of it.
However, I am having a issue that is hindering me from getting started. I have created a Virtualenv called "flask", and I have tried to install flask and some extensions into it via pip. When I try to install flask/extensions, they are all installed with python 2.6 for some reason. I see that in the "flask" lib, there is only Python 2.6. The file structure looks like this:
flask
bin
app
lib
Python2.6
I want them installed with Python 2.7.6, as that is my system version.
python -V
Python 2.7.6
My guess is that I need to remove Python2.6 from the lib and replace it with Python, but I don't want to risk harming my system in anyway. Any help would be much appreciated, and I can provide any other information as necessary.
Thanks
My program takes the first and last name from a csv file row and puts it into a Google search argument along with another word. The search becomes: John Doe teacher I would like it to be: "John Doe" + teacher How can I get the quotes and the + sign? Thanks!
I'm using the following code to split a CSV file into multiple chunks (sourced from here)
def worker(chunk):
print len(chunk)
def keyfunc(row):
return row[0]
def main():
pool = mp.Pool()
largefile = 'Counseling.csv'
num_chunks = 10
start_time = time.time()
results = []
with open(largefile) as f:
reader = csv.reader(f)
reader.next()
chunks = itertools.groupby(reader, keyfunc)
while True:
# make a list of num_chunks chunks
groups = [list(chunk) for key, chunk in
itertools.islice(chunks, num_chunks)]
if groups:
result = pool.map(worker, groups)
results.extend(result)
else:
break
pool.close()
pool.join()
However, it seems that the number of chunks always remains constant regardless of the number of chunks that I choose to use. For example, whether I choose to have 1 or 10 chunks, I always get this output when processing a sample file. Ideally, I'd like to chunk a file so that it is equitably distributed.
Note, the real file I am chunking is over 13 million rows long which is why I am processing it piece by piece. That is a must!
6
7
1
...
1
1
94
--- 0.101687192917 seconds ---
I am using wxPython and matplotlib to create a splitter window that supports commands on one window and plots data on the other. How do I get plt.plot() to attach the plot on the plotting window?
I get a popup window when I run plt.plot() and it does not accept a parent window as an argument. I need to use plt.plot() instead of the backend modules such as wxAgg because it already has the zoom and save file features and the code looks much simpler.
having issues using set_xlim. (possibly because of datetime objects??)
here's my code (doing it in ipython notebook):
%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
import datetime
date_list = [datetime.datetime(2015, 6, 20, 0, 0), datetime.datetime(2015, 6, 21, 0, 0), datetime.datetime(2015, 6, 22, 0, 0), datetime.datetime(2015, 6, 23, 0, 0), datetime.datetime(2015, 6, 24, 0, 0), datetime.datetime(2015, 6, 25, 0, 0), datetime.datetime(2015, 6, 26, 0, 0)]
count_list = [11590, 10743, 27369, 31023, 30569, 31937, 30205]
fig=plt.figure(figsize=(10,3.5))
ax=fig.add_subplot(111)
width = 0.8
tickLocations = np.arange(7)
ax.set_title("Turnstiles Totals for Lexington Station C/A A002 Unit R051 from 6/20/15-6/26-15")
ax.bar(date_list, count_list, width, color='wheat', edgecolor='#8B7E66', linewidth=4.0)
ax.set_xticklabels(date_list, rotation = 315, horizontalalignment = 'left')
This gives me:
But when I try to make some extra space on the leftmost and rightmost edges with this code:
ax.set_xlim(xmin=-0.6, xmax=0.6)
I get this huge error (this is just the bottom snippet):
223 tz = _get_rc_timezone()
224 ix = int(x)
--> 225 dt = datetime.datetime.fromordinal(ix)
226 remainder = float(x) - ix
227 hour, remainder = divmod(24 * remainder, 1)
ValueError: ordinal must be >= 1
Any idea what's going on guys? thanks!
This is a tux_database module, one of 6 modules in which a tux_master module imports and gets rss feed info and send it to my email. I need it to log each process status as follows:
input okay or fail? rss okay or fail? email okay or fail?
Can anyone help me get my database to write status records?
import sqlite3 as lite
import sys
def Data(Status, currentDate):
print("Status Update Time")
try:
con = lite.connect('tux.db')
with con:
cur = con.cursor()
cur.execute("DROP TABLE IF exists EventLookin")
cur.execute("DROP TABLE IF exists EventStatus")
cur.execute("CREATE TABLE EventLookin(EventID TEXT PRIMARY KEY, EventType TEXT, EventDate)")
cur.execute("CREATE TABLE EventStatus(EventID INTEGER PRIMARY KEY, EventType TEXT, EventDate, FOREIGN KEY(EventType) REFERENCES Lookin(EventType) )")
#rssFeed TEXT, emailAddress TEXT, currentDate TEXT , costValue TEXT)")
# cur.execute("CREATE TABLE EventLookin(EventId varchar(50), EventType varchar(50), EventDate varchar(50))")
cur.execute("INSERT INTO EventLookin VALUES(' IF ', 'Input Fail','Date: ')")
cur.execute("INSERT INTO EventLookin VALUES('RO: ','RSS Okay','Date: ' )")
cur.execute("INSERT INTO EventLookin VALUES('EO: ','Email Okay','Date: ' )")
cur.execute("INSERT INTO EventLookin VALUES('EF: ','Email Fail','Date: ' )")
cur.execute("INSERT INTO EventLookin VALUES('IO: ','Input Okay','Date: ' )")
con.commit()
except lite.Error, e:
if con:
con.rollback()
print"Error %s: " % e.args[0]
sys.exit(1)
finally:
if con:
con.close()
def currentDate(EventType, EventDate):
try:
con = lite.connect("tux.db")
with con:
cur = con.cursor()
insertstring = "INSERT INTO EventStatus Values (NULL,'" + EventType + "','" + EventDate + "')"
cur.execute(insertstring)
finally:
if con:
con.close()
I've been trying to use Scrapy to recover the link of the first image of a query in Google Images.
For example, I've been trying to recover the first link for this specific query: Emma Watson - Google Images. To formulate the Xpath I've been using Xpath Helper which is a extension for Google Chrome. The First XPath I tried was the following:
.//*[@id='rg_s']/div[1]/a/@href
returned the following in the extension: Xpath Helper:
http://ift.tt/1IPmJSG
which actually is what I want. Then I'll scrap that url too and get the url for the Image. That's what I'm looking for. But for some reason I tried the Xpath on Scrapy Shell and returns []. Empty!
Tried with other XPath (to the same place):
.//div[@class='rg_di rg_el ivg-i'][1]/a[@class='rg_l']/@href
and still the same answer: []. I don't know what I'm doing wrong. Can you help me with this?
PS. What I use in Scrapy Shell is the following:
>response.xpath(".//*[@id='rg_s']/div[1]/a/@href")
# returned: []
>response.xpath(".//div[@class='rg_di rg_el ivg-i'][1]/a[@class='rg_l']/@href")
# returned: []
Something to add: When I tried to get the Title of the Page, it works.
>response.xpath(".//title/text()").extract()
# returns: [u'emma watson - Google Search']
I want to solve a mixed integer linear program with the following objective function:
J = maximize (f1(x) + f2(x)) subject to constraint: cost(x) <= threshold
where x is the set of selected variables, f1 and f2 are two scoring functions and cost is the cost function.
f2 is a function based on similarity between the selected variables. I don't know how to formulate this function in pulp.
This is my minimal working example in which function f2 is the similarity between two ingredients and I want to add similarity[i][j] to the objective function if j is already in selected variables, but don't know how to do it.
import numpy as np
import pulp
threshold = 200
model = pulp.LpProblem('selection', pulp.LpMaximize)
similarity = np.array([[1., 0.08333333, 0.1, 0., 0., 0.0625],
[0.08333333, 1., 0.33333333,
0., 0.11111111, 0.07692308],
[0.1, 0.33333333, 1., 0.2, 0., 0.09090909],
[0., 0., 0.2, 1., 0., 0.],
[0., 0.11111111, 0., 0., 1., 0.27272727],
[0.0625, 0.07692308, 0.09090909, 0., 0.27272727, 1.]])
ingredients = ['var_%d' % i for i in range(6)]
scores = np.random.randint(1, 3, size=len(ingredients))
costs = np.random.randint(20, 60, len(ingredients))
scores = dict(zip(ingredients, scores))
costs = dict(zip(ingredients, costs))
x = pulp.LpVariable.dict(
'x_%s', ingredients, lowBound=0, upBound=1, cat=pulp.LpInteger)
model += sum([scores[i] * x[i] for i in ingredients])
model += sum([costs[i] * x[i] for i in ingredients]) <= threshold
solver = pulp.solvers.PULP_CBC_CMD()
model.solve(solver)
This code basically considers only static costs (encoded in costs variable). How can I dynamically add similarity costs that are the similarity variable?
I am a beginer with python and decided to give it a go on my raspberry pi. I learned python from a small book that covers the basics, I combined the demo to change the Tkinter window background color with a toggle button and the lighting up a LED to get this code:
#! /usr/bin/env python
import RPi.GPIO as GPIO
GPIO.setmode(GPIO.BOARD); GPIO.setup(7, GPIO.OUT)
GPIO.output(7, False)
from Tkinter import*
window = Tk()
window.title("Relay Button")
window.configure(bg= "green")
btn_end= Button(window, text = "close", command=exit)
def tog():
if ( GPIO.input(7) == True ):
GPIO.output(7, False)
else:
GPIO.output(7, True)
btn_tog=Button( window, text="Switch", command=tog)
btn_end.pack(padx=100, pady=20)
btn_tog.pack(padx=100, pady=20)
window.mainloop()
GPIO.cleanup()
I get the error :
relaybutton.py:3: RuntimeWarning: This channel is already in use,
continuing anyway. Use GPIO.setwarnings(False) to disable warnings.
GPIO.setmode(GPIO.BOARD); GPIO.setup(7, GPIO.OUT)
I have no clue why I get this error, I have at the end the GPIO.cleanup().