python - Why is my dictionary empty? -
newbie question again. i'm getting pairs of information several url, bs4 &urllib create list of list of list of pair. e.g : pair couple (name,value) e.g : list of pairs 1 url e.g : list of list of pairs able classify got in dictionary.
a list of list of pairs should :
list_of_list_of_pairs = [ [ [a,1],[c,2],[e,3] ], [ [g,4],[r,6] ... ], ... ] here code i'm using :
from bs4 import beautifulsoup import csv import urllib.request collections import * def parse_csv(content, delimiter = ';'): ##we use here ";" parse csv because of european way of dealing excel-csv csv_data = [] line in content.split('\n'): csv_data.append( [x.strip() x in line.split( delimiter )] ) # strips spaces return csv_data list_of_list_of_pairs=[] list_of_pairs=[] list_url=parse_csv(open('url.csv','ru').read()) in range(0,len(list_url)) : url=str(list_url[i][0]) ## read url array coming url-csv page=urllib.request.urlopen(url) soup_0 = beautifulsoup(page.read(),"html.parser") restricted_webpage= soup_0.find( "div", {"id":"ingredients"} ) readable_restricted=str(restricted_webpage) soup=beautifulsoup(readable_restricted,"html.parser") trs = soup.find_all('tr') tr in trs: tds = tr.find_all("td") try: #we using "try" because table not formatted. allows program continue after encountering error. nutriments = str(tds[0].get_text().strip()) print(nutriments) # structure $isolate item column in table , converts string. quantity = str(tds[1].get_text().strip()) print(quantity) pair=[nutriments,quantity] list_of_pairs.append(pair) except: print ("bad tr string") continue #this tells computer move on next item after encounters error list_of_list_of_pairs.append([list_of_pairs]) print(list_of_list_of_pairs) dico = defaultdict(list) n,list_of_pairs in enumerate(list_of_list_of_pairs): i,pairs in enumerate(list_of_pairs): if (len(pairs) == 2): cle = pairs[0] val = pairs[1] while (len(dico[cle]) < n): dico[cle].append(-1) dico[cle].append(val) cle in dico: while (len(dico[cle]) < len(list_of_list_of_pairs)): dico[cle].append(-1) print(dico.items()) problem : dico.items() return empty.
i fear list of list of pairs not being read. maybe badly built. draw attention. number of "[" , "]" @ beginning , @ end of list. think that's way many.
that result of print(list_of_list_pairs) :
[[['cendres brutes (%)', '7.4'], ['cellulose brute (%)', '1.6'], ['fibres alimentaires (%)', '6.6'],..., [[['cendres brutes (%)', '7.4'], ['cellulose brute (%)', '1.6'], ['fibres alimentaires (%)', '6.6'], ['matière grasse (%)', '16.0'], ['acide linoléique (%)', '3.1'], ['energie métabolisable (calculée selon nrc85) (kcal/kg)', '3652.5'], ['energie métabolisable (mesurée) (kcal/kg)', '3900.0'], ['humidité (%)', '9.5'] ['arginine (%)', '1.54'], ['l-lysine (%)', '1.15']]], [[['cendres brutes (%)', '7.4'].. .
i think wrong. name cendres brutes (%) can appear 3 times in list (because there 1 in each url). think something's wrong. doesn't explain why dic empty. or maybe related.
any welcomed !
update : i'm trying : unorganized pairs of informations scrapped several urls , organized csv looks :
namekey valurl1 valurl2 valurl3 2.4 5.6 -1 b 1.2 4.2 -1 c -1 -1 val .... a,b,c ... being can read "cendres brutes" , "acide linoléique" ...
update : print(list_of_list_of_pairs) returns :
[[[['cendres brutes (%)', '7.4'], ['cellulose brute (%)', '1.6'], ['fibres alimentaires (%)', '6.6'], ['matière grasse (%)', '16.0'], ['acide linoléique (%)', '3.1'], ['energie métabolisable (calculée selon nrc85) (kcal/kg)', '3652.5'], ['energie métabolisable (mesurée) (kcal/kg)', '3900.0'], ['humidité (%)', '9.5'], ['extrait non azoté (%)', '40.5'], ['oméga 6 (%)', '3.18'], ['protéine brute (%)', '25.0'], ['amidon (%)', '35.5'], ['chlore (%)', '1.43'], ['cuivre (mg/kg)', '15.0'], ['iode (mg/kg)', '2.9'], ['fer (mg/kg)', '167.0'], ['manganèse (mg/kg)', '68.0'], ['zinc (mg/kg)', '242.0'], ['biotine (mg/kg)', '3.13'], ['choline (mg/kg)', '1600.0'], ['acide folique (mg/kg)', '13.9'], ['vitamine (ui/kg)', '32000.0'], ['vitamine b1 thiamine (mg/kg)', '27.5'], ['vitamine b2 riboflavine (mg/kg)', '49.6'], ['vitamine b3 niacine (mg/kg)', '490.0'], ['vitamine b5 acide pantothénique (mg/kg)', '147.8'], ['vitamine b6 pyridoxine (mg/kg)', '77.1'], ['vitamine c (mg/kg)', '200.0'], ['vitamine d3 (ui/kg)', '800.0'], ['vitamine e (mg/kg)', '600.0'], ['arginine (%)', '1.53'], ['lutéine (mg/kg)', '5.0'], ['méthionine cystine (%)', '1.18'], ['taurine (mg/kg)', '2900.0'], ['acide arachidonique (%)', '0.07'], ['cendres brutes (%)', '6.1'], ['biotine (mg/kg)', '2.92'], ['fibres alimentaires (%)', '11.0'], ['dl-méthionine (%)', '0.5'], ['epa/dha (%)', '0.3'], ['matière grasse (%)', '12.0'], ['cellulose brute (%)', '3.7'], ['chlorure de glucosamine (mg/kg)', '495.0'], ['glucosamine plus chondroïtine (mg/kg)', '500.0'], ['l-carnitine (mg/kg)', '100.0'], ['acide linoléique (%)', '2.42'], ['lutéine (mg/kg)', '5.0'], ['energie métabolisable (calculée selon nrc85) (kcal/kg)', '3425.0'], ['energie métabolisable (mesurée) (kcal/kg)', '3584.0'], ['méthionine cystine (%)', '0.92'], ['humidité (%)', '9.5'], ['extrait non azoté (%)', '41.7'], ['oméga 3 (%)', '0.64'], ['oméga 6 (%)', '2.66'], ['phosphore (%)', '0.7'], ['protéine brute (%)', '27.0'], ['amidon (%)', '34.4'], ['taurine (mg/kg)', '3500.0'], ['vitamine (ui/kg)', '29000.0'], ['vitamine c (mg/kg)', '300.0'], ['vitamine e (mg/kg)', '600.0'], ['calcium (%)', '0.9'], ['polyphénols de thé vert et de raisins (mg/kg)', '150.0'], ['chlore (%)', '0.63'], ['cuivre (mg/kg)', '15.0'], ['iode (mg/kg)', '4.8'], ['fer (mg/kg)', '212.0'], ['magnésium (%)', '0.08'], ['manganèse (mg/kg)', '71.0'], ['potassium (%)', '0.7'], ['sélénium (mg/kg)', '0.29'], ['sodium (%)', '0.4'], ['zinc (mg/kg)', '201.0'], ['choline (mg/kg)', '2200.0'], ['acide folique (mg/kg)', '12.9'], ['vitamine b1 thiamine (mg/kg)', '25.7'], ['vitamine b12 cyanocobalamine (mg/kg)', '0.13'], ['vitamine b2 riboflavine (mg/kg)', '46.2'], ['vitamine b3 niacine (mg/kg)', '458.7'], ['vitamine b5 acide pantothénique (mg/kg)', '137.9'], ['vitamine b6 pyridoxine (mg/kg)', '72.0'], ['vitamine d3 (ui/kg)', '800.0'], ['arginine (%)', '1.52'], ['l-lysine (%)', '1.08'], ['acide arachidonique (%)', '0.07'], ['cendres brutes (%)', '5.9'], ['biotine (mg/kg)', '3.07'], ['fibres alimentaires (%)', '7.0'], ['dl-méthionine (%)', '0.65'], ['epa/dha (%)', '0.3'], ['matière grasse (%)', '16.0'], ['cellulose brute (%)', '1.3'], ['chlorure de glucosamine (mg/kg)', '495.0'], ['glucosamine plus chondroïtine (mg/kg)', '500.0'], ['l-carnitine (mg/kg)', '150.0'], ['acide linoléique (%)', '2.88'], ['lutéine (mg/kg)', '5.0'], ['energie métabolisable (calculée selon nrc85) (kcal/kg)', '3716.0'], ['energie métabolisable (mesurée) (kcal/kg)', '3964.0'], ['méthionine cystine (%)', '1.15'], ['humidité (%)', '9.5'], ['extrait non azoté (%)', '37.3'], ['oméga 3 (%)', '0.63'], ['oméga 6 (%)', '3.13'], ['phosphore (%)', '0.6'], ['protéine brute (%)', '30.0'], ['amidon (%)', '31.6'], ['taurine (mg/kg)', '2000.0'], ['vitamine (ui/kg)', '31000.0'], ['vitamine c (mg/kg)', '300.0'], ['vitamine e (mg/kg)', '600.0'], ['calcium (%)', '0.89'], ['polyphénols de thé vert et de raisins (mg/kg)', '150.0'], ['chlore (%)', '0.68'], ['cuivre (mg/kg)', '15.0'], ['iode (mg/kg)', '4.8'], ['fer (mg/kg)', '200.0'], ['magnésium (%)', '0.12'], ['manganèse (mg/kg)', '73.0'], ['potassium (%)', '0.65'], ['sélénium (mg/kg)', '0.27'], ['sodium (%)', '0.4'], ['zinc (mg/kg)', '205.0'], ['choline (mg/kg)', '2400.0'], ['acide folique (mg/kg)', '13.6'], ['vitamine b1 thiamine (mg/kg)', '26.9'], ['vitamine b12 cyanocobalamine (mg/kg)', '0.13'], ['vitamine b2 riboflavine (mg/kg)', '48.5'], ['vitamine b3 niacine (mg/kg)', '481.9'], ['vitamine b5 acide pantothénique (mg/kg)', '144.9'], ['vitamine b6 pyridoxine (mg/kg)', '75.6'], ['vitamine d3 (ui/kg)', '800.0'], ['arginine (%)', '1.54'], ['l-lysine (%)', '1.15']]]] update 2 : print(list_of_list_of_pairs[0][0]) returns :
[['cendres brutes (%)', '7.4'], ['cellulose brute (%)', '1.6'], ['fibres alimentaires (%)', '6.6'], ['matière grasse (%)', '16.0'], ['acide linoléique (%)', '3.1'], ['energie métabolisable (calculée selon nrc85) (kcal/kg)', '3652.5'], ['energie métabolisable (mesurée) (kcal/kg)', '3900.0'], ['humidité (%)', '9.5'], ['extrait non azoté (%)', '40.5'], ['oméga 6 (%)', '3.18'], ['protéine brute (%)', '25.0'], ['amidon (%)', '35.5'], ['chlore (%)', '1.43'], ['cuivre (mg/kg)', '15.0'], ['iode (mg/kg)', '2.9'], ['fer (mg/kg)', '167.0'], ['manganèse (mg/kg)', '68.0'], ['zinc (mg/kg)', '242.0'], ['biotine (mg/kg)', '3.13'], ['choline (mg/kg)', '1600.0'], ['acide folique (mg/kg)', '13.9'], ['vitamine (ui/kg)', '32000.0'], ['vitamine b1 thiamine (mg/kg)', '27.5'], ['vitamine b2 riboflavine (mg/kg)', '49.6'], ['vitamine b3 niacine (mg/kg)', '490.0'], ['vitamine b5 acide pantothénique (mg/kg)', '147.8'], ['vitamine b6 pyridoxine (mg/kg)', '77.1'], ['vitamine c (mg/kg)', '200.0'], ['vitamine d3 (ui/kg)', '800.0'], ['vitamine e (mg/kg)', '600.0'], ['arginine (%)', '1.53'], ['lutéine (mg/kg)', '5.0'], ['méthionine cystine (%)', '1.18'], ['taurine (mg/kg)', '2900.0'], ['acide arachidonique (%)', '0.07'], ['cendres brutes (%)', '6.1'], ['biotine (mg/kg)', '2.92'], ['fibres alimentaires (%)', '11.0'], ['dl-méthionine (%)', '0.5'], ['epa/dha (%)', '0.3'], ['matière grasse (%)', '12.0'], ['cellulose brute (%)', '3.7'], ['chlorure de glucosamine (mg/kg)', '495.0'], ['glucosamine plus chondroïtine (mg/kg)', '500.0'], ['l-carnitine (mg/kg)', '100.0'], ['acide linoléique (%)', '2.42'], ['lutéine (mg/kg)', '5.0'], ['energie métabolisable (calculée selon nrc85) (kcal/kg)', '3425.0'], ['energie métabolisable (mesurée) (kcal/kg)', '3584.0'], ['méthionine cystine (%)', '0.92'], ['humidité (%)', '9.5'], ['extrait non azoté (%)', '41.7'], ['oméga 3 (%)', '0.64'], ['oméga 6 (%)', '2.66'], ['phosphore (%)', '0.7'], ['protéine brute (%)', '27.0'], ['amidon (%)', '34.4'], ['taurine (mg/kg)', '3500.0'], ['vitamine (ui/kg)', '29000.0'], ['vitamine c (mg/kg)', '300.0'], ['vitamine e (mg/kg)', '600.0'], ['calcium (%)', '0.9'], ['polyphénols de thé vert et de raisins (mg/kg)', '150.0'], ['chlore (%)', '0.63'], ['cuivre (mg/kg)', '15.0'], ['iode (mg/kg)', '4.8'], ['fer (mg/kg)', '212.0'], ['magnésium (%)', '0.08'], ['manganèse (mg/kg)', '71.0'], ['potassium (%)', '0.7'], ['sélénium (mg/kg)', '0.29'], ['sodium (%)', '0.4'], ['zinc (mg/kg)', '201.0'], ['choline (mg/kg)', '2200.0'], ['acide folique (mg/kg)', '12.9'], ['vitamine b1 thiamine (mg/kg)', '25.7'], ['vitamine b12 cyanocobalamine (mg/kg)', '0.13'], ['vitamine b2 riboflavine (mg/kg)', '46.2'], ['vitamine b3 niacine (mg/kg)', '458.7'], ['vitamine b5 acide pantothénique (mg/kg)', '137.9'], ['vitamine b6 pyridoxine (mg/kg)', '72.0'], ['vitamine d3 (ui/kg)', '800.0'], ['arginine (%)', '1.52'], ['l-lysine (%)', '1.08'], ['acide arachidonique (%)', '0.07'], ['cendres brutes (%)', '5.9'], ['biotine (mg/kg)', '3.07'], ['fibres alimentaires (%)', '7.0'], ['dl-méthionine (%)', '0.65'], ['epa/dha (%)', '0.3'], ['matière grasse (%)', '16.0'], ['cellulose brute (%)', '1.3'], ['chlorure de glucosamine (mg/kg)', '495.0'], ['glucosamine plus chondroïtine (mg/kg)', '500.0'], ['l-carnitine (mg/kg)', '150.0'], ['acide linoléique (%)', '2.88'], ['lutéine (mg/kg)', '5.0'], ['energie métabolisable (calculée selon nrc85) (kcal/kg)', '3716.0'], ['energie métabolisable (mesurée) (kcal/kg)', '3964.0'], ['méthionine cystine (%)', '1.15'], ['humidité (%)', '9.5'], ['extrait non azoté (%)', '37.3'], ['oméga 3 (%)', '0.63'], ['oméga 6 (%)', '3.13'], ['phosphore (%)', '0.6'], ['protéine brute (%)', '30.0'], ['amidon (%)', '31.6'], ['taurine (mg/kg)', '2000.0'], ['vitamine (ui/kg)', '31000.0'], ['vitamine c (mg/kg)', '300.0'], ['vitamine e (mg/kg)', '600.0'], ['calcium (%)', '0.89'], ['polyphénols de thé vert et de raisins (mg/kg)', '150.0'], ['chlore (%)', '0.68'], ['cuivre (mg/kg)', '15.0'], ['iode (mg/kg)', '4.8'], ['fer (mg/kg)', '200.0'], ['magnésium (%)', '0.12'], ['manganèse (mg/kg)', '73.0'], ['potassium (%)', '0.65'], ['sélénium (mg/kg)', '0.27'], ['sodium (%)', '0.4'], ['zinc (mg/kg)', '205.0'], ['choline (mg/kg)', '2400.0'], ['acide folique (mg/kg)', '13.6'], ['vitamine b1 thiamine (mg/kg)', '26.9'], ['vitamine b12 cyanocobalamine (mg/kg)', '0.13'], ['vitamine b2 riboflavine (mg/kg)', '48.5'], ['vitamine b3 niacine (mg/kg)', '481.9'], ['vitamine b5 acide pantothénique (mg/kg)', '144.9'], ['vitamine b6 pyridoxine (mg/kg)', '75.6'], ['vitamine d3 (ui/kg)', '800.0'], ['arginine (%)', '1.54'], ['l-lysine (%)', '1.15']] it seems list_of_list_of_paris badly built. don't understand why.
since have list of list of pairs, list_of_list_of_pairs[n] still list of pairs, not pairs.
but code assuming list_of_list_of_pairs[n] pair , isn't. should iterate on first list_of_list_of_pairs , elements in (which list of pairs) , try add them dictionary.
example -
for n,list_of_pairs in enumerate(list_of_list_of_pairs): i,pairs in enumerate(list_of_pairs): if (len(pairs) == 2): cle = pairs[0] val = pairs[1] #do rest of logic. also, have loop -
for cle in dico: while (len(dico[cle]) < len(s)): dico[cle].append(-1) but not see s being defined anywhere, s supposed be?
Comments
Post a Comment