如何将 for 循环拆分为 3 个单独的数据框？

Question

Umar

Asked: 2023-11-05 20:34:18 +0800 CST2023-11-05 20:34:18 +0800 CST 2023-11-05 20:34:18 +0800 CST

如何在Python中解析分层数据并将其格式化为TSV文件？

772

我有一个包含分层信息和 KO 编号的数据集，我希望将此数据格式化为 Python 中的 TSV（制表符分隔值）文件，其中第一列包含 KO 编号，第二列包含描述，第三列包含列包含基于输入数据中最近的“A”部分的层次结构。层次结构应包括从“A”、“B”和“C”开始直到最近的“C”部分的元素。此外，如果相同的 KO 编号出现在不同的等级中，则该等级应用 | 分隔。同一行下的输入数据是 file.keg 格式输入数据：

A09100 Metabolism
B
B  09101 Carbohydrate metabolism
C    00010 Glycolysis / Gluconeogenesis [PATH:ko00010]
D      K00844  HK; hexokinase [EC:2.7.1.1]
D      K12407  GCK; glucokinase [EC:2.7.1.2]
D      K00001  E1.1.1.1, adh; alcohol dehydrogenase [EC:1.1.1.1]
B  09103 Lipid metabolism
C    00071 Fatty acid degradation [PATH:ko00071]
D      K00001  E1.1.1.1, adh; alcohol dehydrogenase [EC:1.1.1.1]
A09120 Genetic Information Processing
B
B  09121 Transcription
C    03020 RNA polymerase [PATH:ko03020]
D      K03043  rpoB; DNA-directed RNA polymerase subunit beta [EC:2.7.7.6]
D      K13797  rpoBC; DNA-directed RNA polymerase subunit beta-beta' [EC:2.7.7.6]

预期输出：

    KO      metadata_KEGG_Description        metadata_KEGG_Pathways
K00844  HK; hexokinase [EC:2.7.1.1]     Metabolism, Carbohydrate metabolism, Glycolysis / Gluconeogenesis
K12407  GCK; glucokinase [EC:2.7.1.2]   Metabolism, Carbohydrate metabolism, Glycolysis / Gluconeogenesis
K00001  E1.1.1.1, adh; alcohol dehydrogenase [EC:1.1.1.1]    Metabolism, Carbohydrate metabolism, Glycolysis / Gluconeogenesis|Metabolism, Lipid metabolism, Fatty acid degradation
K03043  rpoB; DNA-directed RNA polymerase subunit beta [EC:2.7.7.6]  Genetic Information Processing, Transcription, RNA polymerase
K13797  rpoBC; DNA-directed RNA polymerase subunit beta-beta' [EC:2.7.7.6]  Genetic Information Processing, Transcription, RNA polymerase

对于如何根据提供的分层信息正确处理这些数据到所需的 TSV 文件中的任何帮助或指导，我将不胜感激。谢谢您的帮助！

这是我的代码

data = """A09100 Metabolism
B
B  09101 Carbohydrate metabolism
C    00010 Glycolysis / Gluconeogenesis [PATH:ko00010]
D      K00844  HK; hexokinase [EC:2.7.1.1]
D      K12407  GCK; glucokinase [EC:2.7.1.2]
D      K00001  E1.1.1.1, adh; alcohol dehydrogenase [EC:1.1.1.1]
B  09103 Lipid metabolism
C    00071 Fatty acid degradation [PATH:ko00071]
D      K00001  E1.1.1.1, adh; alcohol dehydrogenase [EC:1.1.1.1]
A09120 Genetic Information Processing
B
B  09121 Transcription
C    03020 RNA polymerase [PATH:ko03020]
D      K03043  rpoB; DNA-directed RNA polymerase subunit beta [EC:2.7.7.6]
D      K13797  rpoBC; DNA-directed RNA polymerase subunit beta-beta' [EC:2.7.7.6]"""


lines = data.split('\n')

result = []

ko = None
description = None
hierarchy_names = []

for line in lines:
    parts = line.strip().split()
    if parts:
        if parts[0].startswith('A'):
            # Reset hierarchy for a new 'A' section
            hierarchy_names = [" ".join(parts[1:])]
        elif parts[0] == 'K':
            ko = parts[0]
            description = " ".join(parts[1:])
        elif parts[0] == 'D' and len(parts) >= 3:
            ko = parts[1]
            description = " ".join(parts[2:])
        else:
            hierarchy_names.append(" ".join(parts[1:]))

    if ko and description:
        hierarchy_str = ", ".join(hierarchy_names)
        result.append([ko, description, hierarchy_str])

# Add the header row
result.insert(0, ["KO", "metadata_KEGG_Description", "metadata_KEGG_Pathways"])

# Specify the filename for the TSV file
tsv_filename = "output_data.tsv"

with open(tsv_filename, 'w') as tsv_file:
    for row in result:
        tsv_file.write("\t".join(row) + "\n")

print(f"Data saved to {tsv_filename}")

2 个回答

Voted

Timeless · Answer 1 · 2023-11-06T00:24:35+08:00

我建议您检查Orange Bioinformatics处理 KEGG 文件的DBGETEntryParser方法。否则，如果您想在一些正则表达式帮助下使用pandas，您可以尝试以下操作：

import re

with open("file.keg") as f:
    pat = r"^([A-D]) *(\S+)\s*(.+?)\s*(\[.+\])?(?=$)"
    data = re.findall(pat, f.read(), flags=re.MULTILINE)  ^{正则表达式101-演示}

import pandas as pd

tmp = pd.DataFrame(data, columns=["section", "name", "attribute", "path"])
mA = tmp["section"].eq("A"); mD = tmp["section"].eq("D")
df = tmp.assign(entry= tmp["attribute"].where(mA).ffill()).loc[~mA]

parents = (df["entry"].str.cat(df["attribute"].groupby(
        mD.ne(mD.shift()).cumsum(), sort=False)
            .transform(", ".join).where(~mD).ffill(), sep=", ")
            .rename("parents"))

edges = df[["name"]].join(parents).loc[mD, ["parents", "name"]]

out = (df.join(parents).loc[mD].assign(metadata_KEGG_Pathways=
        lambda x: x["attribute"].str.cat(x["path"], sep=" "))
           .groupby("name", sort=False, as_index=False).agg(
               metadata_KEGG_Description=("metadata_KEGG_Pathways", "first"),
               metadata_KEGG_Pathways=("parents", "|".join)))

# out.to_csv("file.tsv", sep="\t", index=False) # uncomment to make a `.tsv`

输出（以表格格式）：

姓名	元数据_KEGG_描述	元数据_KEGG_Pathways
K00844	香港；己糖激酶 [EC:2.7.1.1]	代谢、碳水化合物代谢、糖酵解/糖异生
K12407	GCK；葡萄糖激酶 [EC:2.7.1.2]	代谢、碳水化合物代谢、糖酵解/糖异生
K00001	E1.1.1.1，adh；乙醇脱氢酶 [EC:1.1.1.1]	代谢、碳水化合物代谢、糖酵解/糖异生\|代谢、脂质代谢、脂肪酸降解
K03043	rpoB；DNA 指导的 RNA 聚合酶亚基 β [EC:2.7.7.6]	遗传信息处理、转录、RNA 聚合酶
K13797	rpoBC；DNA 指导的 RNA 聚合酶亚基 beta-beta' [EC:2.7.7.6]	遗传信息处理、转录、RNA 聚合酶

使用networkx进行图形可视化：

import networkx as nx
from itertools import chain, pairwise

G = nx.from_edgelist(
    chain.from_iterable(
        [pairwise(vals) for vals in edges.agg(
            ",".join, axis=1).str.split(",").to_numpy()]),
    create_using=nx.DiGraph
)

Umar · Answer 2 · 2023-11-05T21:44:42+08:00

我得到了这个代码谢谢

data = """A09100 Metabolism
B
B  09101 Carbohydrate metabolism
C    00010 Glycolysis / Gluconeogenesis [PATH:ko00010]
D      K00844  HK; hexokinase [EC:2.7.1.1]
D      K12407  GCK; glucokinase [EC:2.7.1.2]
D      K00001  E1.1.1.1, adh; alcohol dehydrogenase [EC:1.1.1.1]
B  09103 Lipid metabolism
C    00071 Fatty acid degradation [PATH:ko00071]
D      K00001  E1.1.1.1, adh; alcohol dehydrogenase [EC:1.1.1.1]
A09120 Genetic Information Processing
B
B  09121 Transcription
C    03020 RNA polymerase [PATH:ko03020]
D      K03043  rpoB; DNA-directed RNA polymerase subunit beta [EC:2.7.7.6]
D      K13797  rpoBC; DNA-directed RNA polymerase subunit beta-beta' [EC:2.7.7.6]"""

lines = data.split('\n')

result = []

ko = None
description = None
hierarchy_names = []
unique_ko_hierarchy = {}

for line in lines:
    parts = line.strip().split()
    if parts:
        if parts[0].startswith('A'):
            # Reset hierarchy for a new 'A' section
            hierarchy_names = [" ".join(parts[1:])]
        elif parts[0] == 'K':
            ko = parts[0]
            description = " ".join(parts[1:])
        elif parts[0] == 'D' and len(parts) >= 3:
            ko = parts[1]
            description = " ".join(parts[2:])
        else:
            hierarchy_names.append(" ".join(parts[1:]))

    if ko and description:
        hierarchy_str = ", ".join(hierarchy_names)
        if ko in unique_ko_hierarchy:
            unique_ko_hierarchy[ko].append(hierarchy_str)
        else:
            unique_ko_hierarchy[ko] = [hierarchy_str]

# Add the header row
result.append(["KO", "metadata_KEGG_Description", "metadata_KEGG_Pathways"])

for ko, hierarchies in unique_ko_hierarchy.items():
    result.append([ko, description, "|".join(hierarchies)])

# Specify the filename for the TSV file
tsv_filename = "output_data.tsv"

with open(tsv_filename, 'w') as tsv_file:
    for row in result:
        tsv_file.write("\t".join(row) + "\n")

print(f"Data saved to {tsv_filename}")

如何在Python中解析分层数据并将其格式化为TSV文件？

使用 <font color="#xxx"> 突出显示 html 中的代码

为什么在传递 {} 时重载解析更喜欢 std::nullptr_t 而不是类？

您可以使用花括号初始化列表作为（默认）模板参数吗？

为什么列表推导式在内部创建一个函数？

我正在尝试仅使用海龟随机和数学模块来制作吃豆人游戏

java.lang.NoSuchMethodError: 'void org.openqa.selenium.remote.http.ClientConfig.<init>(java.net.URI, java.time.Duration, java.time.Duratio

为什么 'char -> int' 是提升，而 'char -> Short' 是转换（但不是提升）？

为什么库中不调用全局变量的构造函数？

std::common_reference_with 在元组上的行为不一致。哪个是对的？

C++17 中 std::byte 只能按位运算？

如何在Python中解析分层数据并将其格式化为TSV文件？

2 个回答

相关问题