主页 / user-8837764

Brad's questions

Martin Hope
Brad
Asked: 2024-11-19 11:38:13 +0800 CST
  • 5

我有一个 JSON 文件片段，我将其规范化并使用 record_path 和其他列上的元参数，它工作正常，但当我加载整个文件时，我收到一个关键错误。下面是我为片段和完整的 json 下载文件加载编写的代码。

片段作品

import pandas as pd

data = {
    "meta": {
        "disclaimer": "Do not rely on openFDA to make decisions regarding medical care. While we make every effort to ensure that data is accurate, you should assume all results are unvalidated. We may limit or otherwise restrict your access to the API in line with our Terms of Service.",
        "terms": "https://open.fda.gov/terms/",
        "license": "https://open.fda.gov/license/",
        "last_updated": "2024-11-15",
        "results": {
            "skip": 0,
            "limit": 2,
            "total": 118943
        }
    },
    "results": [
        {
            "product_ndc": "73647-062",
            "generic_name": "MENTHOL, CAMPHOR",
            "labeler_name": "Just Brands LLC",
            "brand_name": "JUST CBD - CBD AND THC ULTRA RELIEF",
            "active_ingredients": [
                {
                    "name": "CAMPHOR (SYNTHETIC)",
                    "strength": "2 g/100g"
                },
                {
                    "name": "MENTHOL",
                    "strength": "6 g/100g"
                }
            ],
            "finished": True,
            "packaging": [
                {
                    "package_ndc": "73647-062-04",
                    "description": "113 g in 1 BOTTLE, PUMP (73647-062-04)",
                    "marketing_start_date": "20230314",
                    "sample": False
                }
            ],
            "listing_expiration_date": "20251231",
            "openfda": {
                "manufacturer_name": ["Just Brands LLC"],
                "spl_set_id": ["f664eb79-8897-3a49-e053-2995a90a37b4"],
                "is_original_packager": [True],
                "unii": ["5TJD82A1ET", "L7T10EIP3A"]
            },
            "marketing_category": "OTC MONOGRAPH DRUG",
            "dosage_form": "GEL",
            "spl_id": "16c906dd-6989-9a79-e063-6394a90afa71",
            "product_type": "HUMAN OTC DRUG",
            "route": ["TOPICAL"],
            "marketing_start_date": "20230314",
            "product_id": "73647-062_16c906dd-6989-9a79-e063-6394a90afa71",
            "application_number": "M017",
            "brand_name_base": "JUST CBD - CBD AND THC ULTRA RELIEF"
        },
        {
            "product_ndc": "0591-4039",
            "marketing_end_date": "20250930",
            "generic_name": "CLOBETASOL PROPIONATE",
            "labeler_name": "Actavis Pharma, Inc.",
            "brand_name": "CLOBETASOL PROPIONATE",
            "active_ingredients": [
                {
                    "name": "CLOBETASOL PROPIONATE",
                    "strength": ".05 g/mL"
                }
            ],
            "finished": True,
            "packaging": [
                {
                    "package_ndc": "0591-4039-46",
                    "description": "1 BOTTLE in 1 CARTON (0591-4039-46)  / 59 mL in 1 BOTTLE",
                    "marketing_start_date": "20150828",
                    "marketing_end_date": "20250930",
                    "sample": False
                },
                {
                    "package_ndc": "0591-4039-74",
                    "description": "1 BOTTLE in 1 CARTON (0591-4039-74)  / 125 mL in 1 BOTTLE",
                    "marketing_start_date": "20150828",
                    "marketing_end_date": "20250930",
                    "sample": False
                }
            ],
            "openfda": {
                "manufacturer_name": ["Actavis Pharma, Inc."],
                "rxcui": ["861512"],
                "spl_set_id": ["907e425a-720a-4180-b97c-9e25008a3658"],
                "is_original_packager": [True],
                "unii": ["779619577M"]
            },
            "marketing_category": "NDA AUTHORIZED GENERIC",
            "dosage_form": "SPRAY",
            "spl_id": "33a56b8b-a9a6-4287-bbf4-d68ad0c59e07",
            "product_type": "HUMAN PRESCRIPTION DRUG",
            "route": ["TOPICAL"],
            "marketing_start_date": "20150828",
            "product_id": "0591-4039_33a56b8b-a9a6-4287-bbf4-d68ad0c59e07",
            "application_number": "NDA021835",
            "brand_name_base": "CLOBETASOL PROPIONATE",
            "pharm_class": [
                "Corticosteroid Hormone Receptor Agonists [MoA]",
                "Corticosteroid [EPC]"
            ]
        }
    ]
}

packaging_data = pd.json_normalize(
    data['results'], 
    record_path=["packaging"], 
    meta=['product_ndc', 'brand_name', 'generic_name']
)

active_ingredients_data = pd.json_normalize(
    data['results'], 
    record_path=["active_ingredients"], 
    meta=['product_ndc', 'brand_name', 'generic_name']
)

combined_data = pd.merge(
    packaging_data,
    active_ingredients_data,
    on=['product_ndc', 'brand_name', 'generic_name'],
    how='outer'
)

使用元数据时，JSON 文件的完整下载和加载会出现关键错误

import pandas as pd
import json
import requests, zipfile, io, os

cwd = os.getcwd()
zip_url = 'https://download.open.fda.gov/drug/ndc/drug-ndc-0001-of-0001.json.zip'
r = requests.get(zip_url)
z = zipfile.ZipFile(io.BytesIO(r.content))
z.extractall(cwd)

with open('drug-ndc-0001-of-0001.json', 'r') as file:
    data = json.load(file)

packaging_data = pd.json_normalize(
    data['results'], 
    record_path=["packaging"], 
    meta=['product_ndc', 'brand_name', 'generic_name']
)

active_ingredients_data = pd.json_normalize(
    data['results'], 
    record_path=["active_ingredients"], 
    meta=['product_ndc', 'brand_name', 'generic_name']
)

combined_data = pd.merge(
    packaging_data,
    active_ingredients_data,
    on=['product_ndc', 'brand_name', 'generic_name'],
    how='outer'
)
python
Martin Hope
Brad
Asked: 2024-11-17 00:31:50 +0800 CST
  • 8

这篇文章经过编辑，以获取实际的 JSON 文件（大文件），而不是我提取的示例代码片段（在这篇文章中有效）。我想知道为什么当我在此数据集上使用 record_path 时会出现密钥错误。

在 results 键下有 2 个嵌套键，分别名为 'active_ingredients' 和 'packaging'，当我进行标准化时，我得到

result = pd.json_normalize(data['results'], record_path=["packaging"],meta=['product_ndc'])

预期的列

package_ndc description marketing_start_date    sample  marketing_end_date  product_ndcs

但是当我将 active_ingredients 添加到 record_path 列表时，我收到一个密钥错误。meta 也是如此。当我将其他列（如“brand_name”和“generic_name”）添加到 meta 列表时，我收到一个密钥错误。查看密钥

这不起作用

result = pd.json_normalize(data['results'], record_path=["packaging","active_ingredients"],meta=['product_ndc','brand_name','generic_name'])

感谢您的帮助

这是我用来获取产生关键错误数据的实际代码。

import pandas as pd
import json
import requests, zipfile, io, os

cwd = os.getcwd()
zip_url = 'https://download.open.fda.gov/drug/ndc/drug-ndc-0001-of-0001.json.zip'
r = requests.get(zip_url)
z = zipfile.ZipFile(io.BytesIO(r.content))
z.extractall(cwd)

with open('drug-ndc-0001-of-0001.json', 'r') as file:
    data = json.load(file)

packaging_data = pd.json_normalize(
    data['results'], 
    record_path=["packaging"], 
    meta=['product_ndc', 'brand_name', 'generic_name']
)

active_ingredients_data = pd.json_normalize(
    data['results'], 
    record_path=["active_ingredients"], 
    meta=['product_ndc', 'brand_name', 'generic_name']
)

我将它与您的答案配对并遇到了我在发布问题之前遇到的相同问题。

python
Martin Hope
Brad
Asked: 2024-11-07 03:09:09 +0800 CST
  • 6

我有一个类似于这个虚构的数据集

data = {'category':['foo','foo','foo','foo','foo','bar','bar','bar','bar','bar'],'part':['101','102','103','104','105','201','202','203','204','205'],'site_a':['yes','yes','no','no','no','no','yes','no','no','no'],'site_b':['no','no','no','no','no','no','yes','no','no','no'],'site_c':['yes','no','no','no','no','no','no','no','no','no']}

我需要按类别分组，并筛选出每个类别值全部为“否”的网站。对于类别“foo”，“site_b”将为真，而对于类别“bar”，“site_c”将为真。我想如果我将“是”转换为 1，将“否”转换为 0，那么如果我使用 groupby，我就可以执行某种聚合函数。我也读过关于多索引的文章，但不确定如何遍历类别以查找哪些网站全部为“否”值。

这就是我的多索引的样子

df.set_index(['category','part'], inplace=True)

                      site_a    site_b  site_c
category    part            
foo         101       yes        no      yes
            102       yes        no      no
            103       no         no      no
            104       no         no      no
            105       no         no      no
bar         201       no         no      no
            202       yes        yes     no
            203       no         no      no
            204       no         no      no
            205       no         no      no

这样，如果某个站点没有使用该类别中的某个部分，那么我会从每个站点的类别/部分列表中删除该类别。

非常感谢关于使用 groupby 或 multiindex 的指导

python