{"id":2948,"date":"2025-04-19T17:41:11","date_gmt":"2025-04-19T09:41:11","guid":{"rendered":"https:\/\/www.gnn.club\/?p=2948"},"modified":"2025-06-24T14:23:42","modified_gmt":"2025-06-24T06:23:42","slug":"tutorial-11-%e6%b1%82%e7%9f%a5%e8%8b%a5%e6%b8%b4%ef%bc%9a%e4%b8%bb%e5%8a%a8%e5%ad%a6%e4%b9%a0active-learning","status":"publish","type":"post","link":"http:\/\/www.gnn.club\/?p=2948","title":{"rendered":"Tutorial 11 &#8211; \u6c42\u77e5\u82e5\u6e34\uff1a\u4e3b\u52a8\u5b66\u4e60(Active Learning)"},"content":{"rendered":"<h1><img decoding=\"async\" src=\"https:\/\/gnnclub-1311496010.cos.ap-beijing.myqcloud.com\/wp-content\/uploads\/2025\/04\/20250419170527907.png\" style=\"height:70px;display:inline\"> Deep Learning<\/h1>\n<hr \/>\n<p>create by Deepfinder<\/p>\n<h3><img decoding=\"async\" src=\"https:\/\/img.icons8.com\/bubbles\/50\/000000\/checklist.png\" style=\"height:50px;display:inline\"> Agenda<\/h3>\n<hr \/>\n<ol>\n<li>\u5e08\u5f92\u76f8\u6388\uff1a\u6709\u76d1\u7763\u5b66\u4e60\uff08Supervised Learning\uff09<\/li>\n<li>\u89c1\u5fae\u77e5\u8457\uff1a\u65e0\u76d1\u7763\u5b66\u4e60\uff08Un-supervised Learning\uff09<\/li>\n<li>\u65e0\u5e08\u81ea\u901a\uff1a\u81ea\u76d1\u7763\u5b66\u4e60\uff08Self-supervised Learning\uff09<\/li>\n<li>\u4ee5\u70b9\u5e26\u9762\uff1a\u534a\u76d1\u7763\u5b66\u4e60\uff08Semi-supervised learning\uff09<\/li>\n<li>\u660e\u8fa8\u662f\u975e\uff1a\u5bf9\u6bd4\u5b66\u4e60\uff08Contrastive Learning\uff09<\/li>\n<li>\u4e3e\u4e00\u53cd\u4e09\uff1a\u8fc1\u79fb\u5b66\u4e60\uff08Transfer Learning\uff09<\/li>\n<li>\u9488\u950b\u76f8\u5bf9\uff1a\u5bf9\u6297\u5b66\u4e60\uff08Adversarial Learning\uff09<\/li>\n<li>\u4f17\u5fd7\u6210\u57ce\uff1a\u96c6\u6210\u5b66\u4e60(Ensemble Learning) <\/li>\n<li>\u6b8a\u9014\u540c\u5f52\uff1a\u8054\u90a6\u5b66\u4e60\uff08Federated Learning\uff09<\/li>\n<li>\u767e\u6298\u4e0d\u6320\uff1a\u5f3a\u5316\u5b66\u4e60\uff08Reinforcement Learning\uff09<\/li>\n<li><strong>\u6c42\u77e5\u82e5\u6e34\uff1a\u4e3b\u52a8\u5b66\u4e60\uff08Active Learning\uff09<\/strong><\/li>\n<li>\u4e07\u6cd5\u5f52\u5b97\uff1a\u5143\u5b66\u4e60\uff08Meta-Learning\uff09<\/li>\n<\/ol>\n<h2><img decoding=\"async\" src=\"https:\/\/img.icons8.com\/cute-clipart\/64\/000000\/alarm.png\" style=\"height:50px;display:inline\"> \u4e3b\u52a8\u5b66\u4e60\u7684\u52a8\u673a<\/h2>\n<hr \/>\n<p>\u4e3b\u52a8\u5b66\u4e60\uff08Active Learning\uff09\u662f\u4e00\u79cd\u673a\u5668\u5b66\u4e60\u8303\u5f0f\uff0c\u65e8\u5728\u901a\u8fc7\u9009\u62e9\u6700\u6709\u4ef7\u503c\u7684\u6837\u672c\u8fdb\u884c\u6807\u6ce8\uff0c<strong>\u4ece\u800c\u5728\u51cf\u5c11\u6807\u6ce8\u6210\u672c\u7684\u540c\u65f6\u63d0\u9ad8\u6a21\u578b\u6027\u80fd<\/strong>\u3002<\/p>\n<p>\u4e3b\u52a8\u5b66\u4e60\u7684\u6838\u5fc3\u601d\u60f3\u662f\uff1a<strong>\u8ba9\u6a21\u578b\u4e3b\u52a8\u9009\u62e9\u90a3\u4e9b\u5bf9\u81ea\u8eab\u5b66\u4e60\u6700\u6709\u5e2e\u52a9\u7684\u6837\u672c\u8fdb\u884c\u6807\u6ce8\uff0c\u800c\u4e0d\u662f\u968f\u673a\u9009\u62e9\u6837\u672c<\/strong>\u3002\u901a\u8fc7\u8fd9\u79cd\u65b9\u5f0f\uff0c\u4e3b\u52a8\u5b66\u4e60\u53ef\u4ee5\u5728\u8f83\u5c11\u7684\u6807\u6ce8\u6570\u636e\u4e0b\uff0c\u8bad\u7ec3\u51fa\u6027\u80fd\u66f4\u597d\u7684\u6a21\u578b\u3002<\/p>\n<p>\u652f\u6301\u5411\u91cf\u673a\uff08SVM\uff09\u7684\u6838\u5fc3\u539f\u7406\u662f\u901a\u8fc7\u5c11\u91cf\u7684\u652f\u6301\u5411\u91cf\u6765\u5b9a\u4e49\u51b3\u7b56\u8fb9\u754c\uff0c\u800c\u4e0d\u662f\u4f9d\u8d56\u4e8e\u6574\u4e2a\u6570\u636e\u96c6\u3002\u8fd9\u4e9b\u652f\u6301\u5411\u91cf\u662f\u8ddd\u79bb\u51b3\u7b56\u8fb9\u754c\u6700\u8fd1\u7684\u70b9\uff0c\u5b83\u4eec\u51b3\u5b9a\u4e86\u5206\u7c7b\u5668\u7684\u6700\u7ec8\u5f62\u6001\u3002\u5728\u4e3b\u52a8\u5b66\u4e60\u7684\u80cc\u666f\u4e0b\uff0c\u8fd9\u4e00\u7279\u6027\u4e0e\u4e3b\u52a8\u5b66\u4e60\u7684\u7406\u5ff5\u4e0d\u8c0b\u800c\u5408\u2014\u2014\u4e24\u8005\u90fd\u5f3a\u8c03\u4ece\u5927\u91cf\u6570\u636e\u4e2d\u8bc6\u522b\u51fa\u6700\u5173\u952e\u7684\u4fe1\u606f\u3002<\/p>\n<hr \/>\n<p><strong>\u4e3b\u52a8\u5b66\u4e60\u7684\u57fa\u672c\u7ec4\u6210\u90e8\u5206\uff08Basic components of Active Learning\uff09<\/strong><\/p>\n<ul>\n<li>\n<p>\u672a\u6807\u8bb0\u6570\u636e\u6c60\uff08Unlabeled Data Pool\uff09\uff1a\u5927\u91cf\u672a\u6807\u8bb0\u7684\u6570\u636e\uff0c\u662f\u4e3b\u52a8\u5b66\u4e60\u7684\u57fa\u7840\u3002<\/p>\n<\/li>\n<li>\n<p>\u5df2\u6807\u8bb0\u6570\u636e\u96c6\uff08Labeled Data Set\uff09\uff1a\u521d\u59cb\u53ef\u80fd\u4e3a\u7a7a\uff0c\u968f\u7740\u5b66\u4e60\u7684\u8fdb\u884c\u9010\u6e10\u589e\u52a0\u3002<\/p>\n<\/li>\n<li>\n<p>\u5b66\u4e60\u5668\uff08Learner\uff09\uff1a\u4ece\u5df2\u6807\u8bb0\u6570\u636e\u4e2d\u5b66\u4e60\u6a21\u578b\u3002<\/p>\n<\/li>\n<li>\n<p>\u9884\u8a00\u673a\uff08Oracle\uff09\uff1a\u751f\u6210\u771f\u5b9e\u6807\u7b7e\u7684\u8fc7\u7a0b\uff0c\u901a\u5e38\u7531\u4eba\u7c7b\u6807\u6ce8\u5458\u5b8c\u6210\u3002<\/p>\n<\/li>\n<\/ul>\n<p><strong>\u4e3b\u52a8\u5b66\u4e60\u7684\u5e38\u7528\u65b9\u6cd5<\/strong><\/p>\n<ul>\n<li>\n<p><strong>\u4e0d\u786e\u5b9a\u6027\u91c7\u6837\uff08Uncertainty Sampling\uff09<\/strong>\uff1a \u9009\u62e9\u6a21\u578b\u6700\u4e0d\u786e\u5b9a\u7684\u6837\u672c\u8fdb\u884c\u6807\u6ce8\u3002<\/p>\n<\/li>\n<li>\n<p>\u5e38\u7528\u6807\u51c6\uff1a<\/p>\n<p>\u25cb \u6700\u5c0f\u7f6e\u4fe1\u5ea6\u5206\u6570\uff08Least Confidence\uff09\uff1a\u9009\u62e9\u6a21\u578b\u6700\u4e0d\u786e\u5b9a\u7684\u6837\u672c\u3002<\/p>\n<p>\u25cb \u6700\u5c0f\u95f4\u9694\u5206\u6570\uff08Minimum Margin\uff09\uff1a\u9009\u62e9\u6a21\u578b\u5728\u4e24\u4e2a\u6700\u53ef\u80fd\u7c7b\u522b\u4e4b\u95f4\u72b9\u8c6b\u4e0d\u51b3\u7684\u6837\u672c\u3002<\/p>\n<p>\u25cb \u71b5\u5206\u6570\uff08Entropy\uff09\uff1a\u9009\u62e9\u6a21\u578b\u9884\u6d4b\u71b5\u6700\u9ad8\u7684\u6837\u672c\u3002<\/p>\n<\/li>\n<li>\n<p><strong>\u59d4\u5458\u4f1a\u67e5\u8be2\uff08Query-by-Committee, QBC\uff09<\/strong>\uff1a \u4f7f\u7528\u591a\u4e2a\u6a21\u578b\uff08\u59d4\u5458\u4f1a\uff09\u7684\u9884\u6d4b\u5206\u6b67\u6765\u9009\u62e9\u6837\u672c\u3002<\/p>\n<\/li>\n<li>\n<p>\u5e38\u7528\u6807\u51c6\uff1a<\/p>\n<p>\u25cb \u6295\u7968\u71b5\uff08Vote Entropy\uff09\uff1a\u57fa\u4e8e\u59d4\u5458\u4f1a\u6210\u5458\u7684\u6295\u7968\u7ed3\u679c\u8ba1\u7b97\u71b5\u3002<\/p>\n<p>\u25cb \u5171\u8bc6\u71b5\uff08Consensus Entropy\uff09\uff1a\u57fa\u4e8e\u59d4\u5458\u4f1a\u6210\u5458\u7684\u6982\u7387\u4f30\u8ba1\u8ba1\u7b97\u71b5\u3002<\/p>\n<p>\u25cb KL \u6563\u5ea6\uff08Kullback-Leibler Divergence\uff09\uff1a\u8861\u91cf\u59d4\u5458\u4f1a\u6210\u5458\u4e0e\u6574\u4f53\u5171\u8bc6\u7684\u5206\u6b67\u3002<\/p>\n<\/li>\n<li>\n<p><strong>\u9884\u671f\u6a21\u578b\u53d8\u5316\uff08Expected Model Change, EMC\uff09<\/strong>\uff1a <\/p>\n<p>\u25cb \u9009\u62e9\u90a3\u4e9b\u80fd\u591f\u6700\u5927\u7a0b\u5ea6\u6539\u53d8\u6a21\u578b\u7684\u6837\u672c\u3002<\/p>\n<p>\u25cb \u901a\u8fc7\u8ba1\u7b97\u68af\u5ea6\u7684\u9884\u671f\u957f\u5ea6\u6765\u8fd1\u4f3c\u6a21\u578b\u53d8\u5316\u3002<\/p>\n<\/li>\n<li>\n<p><strong>\u9884\u671f\u8bef\u5dee\u51cf\u5c11\uff08Expected Error Reduction, EER\uff09<\/strong>\uff1a<\/p>\n<p>\u25cb \u9009\u62e9\u90a3\u4e9b\u80fd\u591f\u6700\u5927\u7a0b\u5ea6\u51cf\u5c11\u6a21\u578b\u5728\u672a\u6765\u6570\u636e\u4e0a\u8bef\u5dee\u7684\u6837\u672c\u3002<\/p>\n<p>\u25cb \u9700\u8981\u91cd\u65b0\u8bad\u7ec3\u6a21\u578b\uff0c\u8ba1\u7b97\u6210\u672c\u8f83\u9ad8\u3002<\/p>\n<\/li>\n<li>\n<p><strong>\u5bc6\u5ea6\u52a0\u6743\uff08Density Weighting\uff09<\/strong>\uff1a<\/p>\n<p>\u25cb \u5728\u9009\u62e9\u6837\u672c\u65f6\uff0c\u4e0d\u4ec5\u8003\u8651\u6a21\u578b\u7684\u4e0d\u786e\u5b9a\u6027\uff0c\u8fd8\u8003\u8651\u6837\u672c\u5728\u6570\u636e\u5206\u5e03\u4e2d\u7684\u5bc6\u5ea6\u3002<\/p>\n<p>\u25cb \u907f\u514d\u9009\u62e9\u7a00\u758f\u533a\u57df\u6216\u5f02\u5e38\u503c\u7684\u6837\u672c\u3002<\/p>\n<\/li>\n<\/ul>\n<pre><code class=\"language-python\">\n%matplotlib inline\nimport numpy as np\nimport matplotlib.pyplot as plt\nfrom scipy.stats import multivariate_normal, ortho_group\nfrom sklearn.linear_model import LogisticRegression\nimport seaborn as sns\nfrom typing import Callable, Sequence, Tuple, Optional\nfrom random import choice\n\n# \u8bbe\u7f6e Seaborn \u6837\u5f0f\nsns.set_style(&quot;white&quot;)\ncolors = sns.color_palette(n_colors=10)<\/code><\/pre>\n<p><strong>\u4ece\u4e24\u4e2a\u9ad8\u65af\u5206\u5e03\u4e2d\u62bd\u53d6\u4e8c\u7ef4\u6570\u636e\u6837\u672c<\/strong>\uff0c\u8c03\u6574\u79cd\u5b50\u4ee5\u83b7\u5f97\u4e0d\u540c\u5f62\u72b6\u7684\u9ad8\u65af\u5206\u5e03\u3002<\/p>\n<pre><code class=\"language-python\"># \u5b9a\u4e49\u8f85\u52a9\u51fd\u6570\ndef sample_covar(dim: int = 2, scale_min: float = 0.3, scale_max: float = 5) -&gt; np.ndarray:\n    &quot;&quot;&quot;Construct random covariance matrix by sampling a rotation and scale matrix.&quot;&quot;&quot;\n    R = ortho_group.rvs(dim) * (np.random.rand(dim, dim) * (scale_max - scale_min) + scale_min)\n    return R.dot(R.T)\n\n# \u751f\u6210\u793a\u4f8b\u6570\u636e\nN_1, N_2 = 100, 100  # number of samples in each point cloud\nmu_1 = [-5, 2]  # centroid of the first point cloud\nmu_2 = [5, -2]  # centroid of the second point cloud\n\nnp.random.seed(27)\n\nX = np.concatenate([\n    multivariate_normal(mu_1, sample_covar()).rvs(N_1),\n    multivariate_normal(mu_2, sample_covar()).rvs(N_2),\n])\ny = np.concatenate([np.zeros(N_1, int), np.ones(N_2, int)])\n\n# \u663e\u793a\u6570\u636e\u7684\u6563\u70b9\u56fe\nplt.figure(figsize=(10, 8))\nplt.scatter(X[y == 0, 0], X[y == 0, 1], color=colors[0])\nplt.scatter(X[y == 1, 0], X[y == 1, 1], color=colors[1])\nplt.xticks(()), plt.yticks(())\nplt.show()\n<\/code><\/pre>\n<p align=\"center\">\n  <img decoding=\"async\" src=\"https:\/\/gnnclub-1311496010.cos.ap-beijing.myqcloud.com\/wp-content\/uploads\/2025\/04\/20250419173502680.png\" style=\"height:500px\">\n<\/p>\n<h3>\u7528\u4e8e\u7ed8\u5236\u6570\u636e\u548c\u5206\u7c7b\u5668\u7684\u8f85\u52a9\u51fd\u6570(\u4e0d\u91cd\u8981)<\/h3>\n<pre><code class=\"language-python\"># \u5b9a\u4e49\u8f85\u52a9\u51fd\u6570\ndef meshgrid_from_bounds(lower_left: Sequence[float], upper_right: Sequence[float], *,\n                         resolution: float = 0.02) -&gt; Tuple[np.ndarray, np.ndarray]:\n    &quot;&quot;&quot;Return a meshgrid that covers the range from lower_left to upper_right.&quot;&quot;&quot;\n    return np.meshgrid(np.arange(lower_left[0], upper_right[0], resolution),\n                       np.arange(lower_left[1], upper_right[1], resolution))\n\ndef plot_decision_boundary(clf, lower_left: Sequence[float], upper_right: Sequence[float], *,\n                           resolution: float = 0.02, boundary_color: Tuple[float] = (0.2, 0.2, 0.2)):\n    &quot;&quot;&quot;Plot decision boundary of a given classifier in the given range.&quot;&quot;&quot;\n    xx, yy = meshgrid_from_bounds(lower_left, upper_right, resolution=resolution)\n    Z = clf.predict_proba(np.c_[xx.ravel(), yy.ravel()])[:, 0].reshape(xx.shape)\n    plt.contour(xx, yy, Z, levels=[0.5], colors=[boundary_color])\n\ndef lighten(color: Tuple[float], *, amount: float = 0.6):\n    &quot;&quot;&quot;Blend color with white by the given amount.&quot;&quot;&quot;\n    return tuple(np.array(color) * (1 - amount) + np.array([1, 1, 1]) * amount)\n\ndef plot_data(X: np.ndarray, y: np.ndarray, train_idx: Sequence[int], *,\n              colors: Sequence[Tuple[float]] = sns.color_palette(n_colors=2)):\n    &quot;&quot;&quot;Scatter plot of the training and unlabeled data.&quot;&quot;&quot;\n    # \u663e\u793a\u6240\u6709\u6570\u636e\n    plt.scatter(X[y == 0, 0], X[y == 0, 1], color=lighten(colors[0]), s=6)\n    plt.scatter(X[y == 1, 0], X[y == 1, 1], color=lighten(colors[1]), s=6)\n\n    # \u663e\u793a\u8bad\u7ec3\u6570\u636e\n    X_tr = X[train_idx]\n    y_tr = y[train_idx]\n    plt.scatter(X_tr[y_tr == 0, 0], X_tr[y_tr == 0, 1], color=colors[0])\n    plt.scatter(X_tr[y_tr == 1, 0], X_tr[y_tr == 1, 1], color=colors[1])\n    plt.xticks(()), plt.yticks(())\n\ndef plot_data_and_classifier(X: np.ndarray, y: np.ndarray, train_idx: Sequence[int], clf, *,\n                             previous_clf=None, colors: Sequence[Tuple[float]] = sns.color_palette(n_colors=2)):\n    &quot;&quot;&quot;Scatter plot of the training and unlabeled data and the decision boundary of the classifier.&quot;&quot;&quot;\n    if previous_clf is not None:\n        plot_decision_boundary(previous_clf, lower_left=X.min(axis=0) - 1, upper_right=X.max(axis=0) + 1, boundary_color=(0.7, 0.7, 0.7))\n\n    plot_decision_boundary(clf, lower_left=X.min(axis=0) - 1, upper_right=X.max(axis=0) + 1)\n    plot_data(X, y, train_idx, colors=colors)\n<\/code><\/pre>\n<h2>\u975e\u4e3b\u52a8\u5b66\u4e60<\/h2>\n<pre><code class=\"language-python\"># \u521d\u59cb\u5316\u968f\u673a\u79cd\u5b50\nnp.random.seed(42)  # \u9009\u62e9\u4e00\u4e2a\u7279\u5b9a\u7684\u968f\u673a\u79cd\u5b50\n\n# \u4ece\u6bcf\u4e00\u7c7b\u4e2d\u968f\u673a\u9009\u62e9\u4e00\u4e2a\u6837\u672c\u4f5c\u4e3a\u521d\u59cb\u8bad\u7ec3\u96c6\nstarting_samples = [choice(range(N_1)), N_1 + choice(range(N_2))]\ntrain_idx = starting_samples[:]\nclf = LogisticRegression().fit(X[train_idx], y[train_idx])\n\n# \u7ed8\u5236\u521d\u59cb\u8bad\u7ec3\u96c6\u548c\u5206\u7c7b\u5668\nplt.figure(figsize=(15, 12))\nplt.subplot(3, 3, 1)\nplt.title(&#039;Initial Training Set&#039;)\nplot_data_and_classifier(X, y, train_idx, clf)\n\n# \u8fdb\u884c8\u8f6e\u968f\u673a\u91c7\u6837\nfor round_num in range(8):  \n    new_sample = choice([i for i in range(N_1 + N_2) if i not in train_idx])\n    train_idx.append(new_sample)\n    previous_clf = clf\n    clf = LogisticRegression().fit(X[train_idx], y[train_idx])\n\n    ax = plt.subplot(3, 3, round_num + 2)\n    plt.title(f&#039;Round {round_num + 1} of 8&#039;)\n    plot_data_and_classifier(X, y, train_idx, clf, previous_clf=previous_clf)\n\nplt.tight_layout()\nplt.show()<\/code><\/pre>\n<p align=\"center\">\n  <img decoding=\"async\" src=\"https:\/\/gnnclub-1311496010.cos.ap-beijing.myqcloud.com\/wp-content\/uploads\/2025\/04\/20250419173628618.png\" style=\"height:700px\">\n<\/p>\n<h2>\u4e0d\u786e\u5b9a\u6027\u91c7\u6837\uff08Uncertainty Sampling\uff09<\/h2>\n<p>\u6700\u7b80\u5355\u7684\u601d\u8def\u662f\uff1a\u6a21\u578b\u8d8a\u4e0d\u786e\u5b9a\u7684\u6837\u672c\uff0c\u8d8a\u503c\u5f97\u6807\u6ce8\u3002\u8fd9\u79cd\u65b9\u6cd5\u53eb\u505a\u4e0d\u786e\u5b9a\u6027\u91c7\u6837\uff0c\u9002\u7528\u4e8e\u4efb\u4f55\u80fd\u591f\u8bc4\u4f30\u81ea\u5df1\u9884\u6d4b\u4e0d\u786e\u5b9a\u6027\u7684\u6a21\u578b\u3002\u6bd4\u5982\uff0c\u6709\u4e9b\u6a21\u578b\u53ef\u4ee5\u7ed9\u51fa\u6bcf\u4e2a\u7c7b\u522b\u7684\u6982\u7387 $\\hat{P}(y \\mid x)$ \uff0c\u6211\u4eec\u53ef\u4ee5\u7528\u6700\u53ef\u80fd\u7684\u7c7b\u522b\u7684\u6982\u7387\u6765\u8861\u91cf\u4e0d\u786e\u5b9a\u6027\uff0c\u516c\u5f0f\u662f\uff1a<\/p>\n<p>$$<br \/>\nu(x)=1-\\max _y \\hat{P}(y \\mid x)<br \/>\n$$<\/p>\n<p>\u8fd9\u4e2a\u6807\u51c6\u53eb\u505a\u6700\u5c0f\u7f6e\u4fe1\u5ea6\u5206\u6570\uff0c\u610f\u601d\u662f\u9009\u62e9\u90a3\u4e9b\u6a21\u578b\u6700\uff02\u6ca1\u628a\u63e1\uff02\u7684\u6837\u672c\uff0c\u4e5f\u5c31\u662f\u6240\u6709\u7c7b\u522b\u7684\u6982\u7387\u90fd\u5dee\u4e0d\u591a\u7684\u60c5\u51b5\u3002<\/p>\n<p><strong>\u6700\u5c0f\u95f4\u9694\u5206\u6570\uff08Minimum Margin Score\uff09<\/strong><\/p>\n<p>\u9664\u4e86\u6700\u5c0f\u7f6e\u4fe1\u5ea6\u5206\u6570\uff0c\u8fd8\u6709\u4e00\u79cd\u60c5\u51b5\u4e5f\u503c\u5f97\u5173\u6ce8\uff1a\u5f53\u6a21\u578b\u53ea\u80fd\u6392\u9664\u4e00\u90e8\u5206\u7c7b\u522b\uff0c\u4f46\u5269\u4e0b\u7684\u4e24\u4e2a\u7c7b\u522b\u6982\u7387\u51e0\u4e4e\u76f8\u7b49\u65f6\u3002\u8fd9\u65f6\u5019\u53ef\u4ee5\u7528\u6700\u5c0f\u95f4\u9694\u5206\u6570\uff1a<\/p>\n<p>$$<br \/>\nu(x)=\\hat{P}\\left(y_2* \\mid x\\right)-\\hat{P}\\left(y_1* \\mid x\\right)<br \/>\n$$<\/p>\n<p>\u5176\u4e2d $y_1*$ \u548c $y_2*$  \u5206\u522b\u662f\u6a21\u578b\u8ba4\u4e3a\u6700\u53ef\u80fd\u548c\u7b2c\u4e8c\u53ef\u80fd\u7684\u7c7b\u522b\u3002\u8fd9\u4e2a\u6807\u51c6\u4f1a\u9009\u62e9\u90a3\u4e9b\u6a21\u578b\u5728\u4e24\u4e2a\u7c7b\u522b\u4e4b\u95f4\uff02\u72b9\u8c6b\u4e0d\u51b3\uff02\u7684\u6837\u672c\uff0c\u901a\u5e38\u8fd9\u4e9b\u6837\u672c\u4f4d\u4e8e\u4e24\u4e2a\u7c7b\u522b\u7684\u5206\u754c\u7ebf\u9644\u8fd1\u3002<\/p>\n<p><strong>\u71b5\u5206\u6570\uff08Entropy Score\uff09<\/strong><\/p>\n<p>\u7b2c\u4e09\u4e2a\u6807\u51c6\u6765\u81ea\u4fe1\u606f\u8bba\uff0c\u53eb\u505a\u5ae1\u5206\u6570\u3002\u71b5\u53ef\u4ee5\u7406\u89e3\u4e3a\uff02\u6df7\u4e71\u7a0b\u5ea6\uff02\uff0c\u5728\u8fd9\u91cc\u8868\u793a\u6a21\u578b\u5bf9\u6837\u672c\u7c7b\u522b\u7684\u4e0d\u786e\u5b9a\u6027\u3002\u516c\u5f0f\u662f\uff1a<\/p>\n<p>$$<br \/>\nu(x)=\\mathbb{E}[-\\log \\hat{P}(y \\mid x)]<br \/>\n$$<\/p>\n<p>\u7b80\u5355\u6765\u8bf4\uff0c\u71b5\u5206\u6570\u8861\u91cf\u7684\u662f\u6a21\u578b\u5728\u77e5\u9053\u6837\u672c\u6807\u7b7e\u540e\u4f1a\u83b7\u5f97\u591a\u5c11\u4fe1\u606f\u3002\u5982\u679c\u6a21\u578b\u5bf9\u67d0\u4e2a\u6837\u672c\u7684\u7c7b\u522b\u975e\u5e38\u786e\u5b9a\uff0c\u71b5\u5206\u6570\u4f1a\u5f88\u4f4e\uff1b\u5982\u679c\u6a21\u578b\u5bf9\u7c7b\u522b\uff02\u62ff\u4e0d\u51c6\uff02\uff0c\u71b5\u5206\u6570\u4f1a\u5f88\u9ad8\u3002\u8fd9\u4e2a\u6807\u51c6\u548c\u6700\u5c0f\u95f4\u9694\u5206\u6570\u6709\u70b9\u50cf\uff0c\u4f46\u5b83\u4f1a\u540c\u65f6\u8003\u8651\u6240\u6709\u7c7b\u522b\uff0c\u800c\u4e0d\u4ec5\u4ec5\u662f\u524d\u4e24\u4e2a\u6700\u53ef\u80fd\u7684\u7c7b\u522b\u3002<\/p>\n<p>\u901a\u4fd7\u6765\u8bf4\uff0c\u4e0d\u786e\u5b9a\u6027\u91c7\u6837\u7684\u6838\u5fc3\u601d\u60f3\u662f\uff1a<strong>\u8ba9\u6a21\u578b\u81ea\u5df1\u51b3\u5b9a\u54ea\u4e9b\u6837\u672c\u6700\u503c\u5f97\u6807\u6ce8<\/strong>\u3002 \u901a\u8fc7\u9009\u62e9\u6a21\u578b\u6700\u4e0d\u786e\u5b9a\u7684\u6837\u672c\uff0c\u6211\u4eec\u53ef\u4ee5\u7528\u66f4\u5c11\u7684\u6807\u6ce8\u6570\u636e\u8bad\u7ec3\u51fa\u66f4\u597d\u7684\u6a21\u578b\u3002<\/p>\n<pre><code class=\"language-python\">from scipy.stats import entropy\nimport numpy as np\nimport matplotlib.pyplot as plt\nfrom sklearn.linear_model import LogisticRegression\nfrom typing import Callable, Sequence\n\n# \u5b9a\u4e49\u4e0d\u786e\u5b9a\u6027\u91c7\u6837\u51c6\u5219\ndef least_confidence(posterior: np.ndarray) -&gt; np.ndarray:\n    &quot;&quot;&quot;Least confidence criterion, u(x) = 1 - max P(y | x).&quot;&quot;&quot;\n    return 1 - posterior.max(axis=1)\n\ndef minimum_margin(posterior: np.ndarray) -&gt; np.ndarray:\n    &quot;&quot;&quot;Minimum margin criterion, u(x) = P(y_2* | x) - P(y_1* | x), where y_1* and y_2* are the two most probable classes.&quot;&quot;&quot;\n    most_probable = posterior.max(axis=1)\n    mask_most_probable = (posterior == most_probable.reshape(-1, 1))\n    second_most_probable = (posterior - mask_most_probable).max(axis=1)\n    return second_most_probable - most_probable\n\ndef entropy_criterion(posterior: np.ndarray) -&gt; np.ndarray:\n    &quot;&quot;&quot;Entropy criterion, u(x) = -E[log P(y | x)].&quot;&quot;&quot;\n    return entropy(posterior, axis=1)\n\ndef uncertainty_sampling(clf, X: np.ndarray, train_idx: Sequence[int], criterion: Callable[[np.ndarray], np.ndarray]) -&gt; int:\n    &quot;&quot;&quot;Get the index of an unknown sample where the classifier is most uncertain about the label.&quot;&quot;&quot;\n    unknown_indices = np.array([i for i in range(X.shape[0]) if i not in train_idx])\n    posterior = clf.predict_proba(X[unknown_indices])\n    utility = criterion(posterior)\n    return unknown_indices[np.argmax(utility)]\n\n# \u521d\u59cb\u5316\u8bad\u7ec3\u96c6\u548c\u5206\u7c7b\u5668\ntrain_idx = starting_samples[:]\nclf = LogisticRegression().fit(X[train_idx], y[train_idx])\n\n# \u7ed8\u5236\u521d\u59cb\u8bad\u7ec3\u96c6\u548c\u5206\u7c7b\u5668\nplt.figure(figsize=(15, 12))\nplt.subplot(3, 3, 1)\nplt.title(&#039;Initial Training Set&#039;)\nplot_data_and_classifier(X, y, train_idx, clf)\n\n# \u8fdb\u884c8\u8f6e\u4e0d\u786e\u5b9a\u6027\u91c7\u6837\nfor round_num in range(8):  # \u907f\u514d\u4e0e\u5185\u7f6e\u51fd\u6570 round \u51b2\u7a81\n    new_sample = uncertainty_sampling(clf, X, train_idx, criterion=entropy_criterion)\n    train_idx.append(new_sample)\n\n    plt.subplot(3, 3, round_num + 2)\n    plt.title(f&#039;Round {round_num + 1} of 8&#039;)\n    previous_clf = clf\n    clf = LogisticRegression().fit(X[train_idx], y[train_idx])\n    plot_data_and_classifier(X, y, train_idx, clf, previous_clf=previous_clf)\n\nplt.tight_layout()\nplt.show()<\/code><\/pre>\n<p align=\"center\">\n  <img decoding=\"async\" src=\"https:\/\/gnnclub-1311496010.cos.ap-beijing.myqcloud.com\/wp-content\/uploads\/2025\/04\/20250419173718173.png\" style=\"height:700px\">\n<\/p>\n<h2>\u59d4\u5458\u4f1a\u67e5\u8be2<\/h2>\n<p>\u59d4\u5458\u4f1a\u67e5\u8be2\uff08QBC\uff09\u7684\u7075\u611f\u6765\u81ea\u4e8e\u96c6\u6210\u5b66\u4e60\u65b9\u6cd5\u3002\u5b83\u4e0d\u662f\u53ea\u7528\u4e00\u4e2a\u5206\u7c7b\u5668\uff0c\u800c\u662f\u8003\u8651\u4e00\u4e2a\u7531\u591a\u4e2a\u5206\u7c7b\u5668 $C=h_1, \\ldots, h_C$ \u7ec4\u6210\u7684\uff02\u59d4\u5458\u4f1a\uff02\u7684\u51b3\u7b56\u3002\u6bcf\u4e2a\u5206\u7c7b\u5668\u7684\u76ee\u6807\u7c7b\u522b\u76f8\u540c\uff0c\u4f46\u5b83\u4eec\u7684\u5e95\u5c42\u6a21\u578b\u6216\u5bf9\u6570\u636e\u7684\u89c6\u89d2\u4e0d\u540c\u3002\u968f\u673a\u68ee\u6797\u5c31\u662f\u4e00\u4e2a\u5178\u578b\u7684\u4f8b\u5b50\uff1a\u6bcf\u4e2a\u5206\u7c7b\u5668\u90fd\u662f\u51b3\u7b56\u6811\uff0c\u4f46\u6bcf\u68f5\u6811\u662f\u57fa\u4e8e\u4e0d\u540c\u7684\u7279\u5f81\u5b50\u96c6\u8bad\u7ec3\u7684\u3002<\/p>\n<h4><img decoding=\"async\" src=\"https:\/\/img.icons8.com\/?size=100&id=91CnU00i6HLv&format=png&color=000000\" style=\"height:50px;display:inline\">  \u5982\u4f55\u5224\u65ad\u672a\u6807\u8bb0\u6837\u672c\u7684\u4ef7\u503c\uff1f<\/h4>\n<p>\u5f88\u7b80\u5355\uff1a\u5982\u679c\u59d4\u5458\u4f1a\u6210\u5458\u5bf9\u67d0\u4e2a\u6837\u672c\u7684\u7c7b\u522b\u610f\u89c1\u4e00\u81f4\uff0c\u90a3\u5c31\u4e0d\u9700\u8981\u518d\u6807\u6ce8\u8fd9\u4e2a\u6837\u672c\u4e86\uff1b\u5982\u679c\u59d4\u5458\u4f1a\u6210\u5458\u610f\u89c1\u4e0d\u4e00\u81f4\uff0c\u90a3\u5c31\u9700\u8981\u5411\u9884\u8a00\u673a oracle\uff08\u6bd4\u5982\u4eba\u7c7b\u6807\u6ce8\u5458\uff09\u8be2\u95ee\u8fd9\u4e2a\u6837\u672c\u7684\u6807\u7b7e\u3002\u6362\u53e5\u8bdd\u8bf4\uff0c\u59d4\u5458\u4f1a\u5bf9\u67d0\u4e2a\u6837\u672c\u7684\u5206\u6b67\u7a0b\u5ea6\u53ef\u4ee5\u4ee3\u8868\u8fd9\u4e2a\u6837\u672c\u7684\u4ef7\u503c\u3002<\/p>\n<h4><img decoding=\"async\" src=\"https:\/\/img.icons8.com\/?size=100&id=91CnU00i6HLv&format=png&color=000000\" style=\"height:50px;display:inline\"> \u5982\u4f55\u8861\u91cf\u5206\u6b67\uff1f<\/h4>\n<p>\u5982\u679c\u59d4\u5458\u4f1a\u6709\u5f88\u591a\u6210\u5458\uff0c\u6211\u4eec\u53ef\u4ee5\u901a\u8fc7\u6bcf\u4e2a\u7c7b\u522b $y$ \u7684\u5f97\u7968\u6570 $V(y)$ \u6765\u4f30\u8ba1\u7c7b\u522b\u7684\u6982\u7387\uff0c\u516c\u5f0f\u662f\uff1a<\/p>\n<p>$$<br \/>\n\\hat{P}(y \\mid x) \\approx \\frac{V(y)}{C}<br \/>\n$$<\/p>\n<p>\u4e0d\u8fc7\uff0c\u59d4\u5458\u4f1a\u4e2d\u7684\u5206\u7c7b\u5668\u6570\u91cf $C$ \u901a\u5e38\u4e0d\u591f\u591a\uff0c\u65e0\u6cd5\u5f97\u5230\u53ef\u9760\u7684\u6982\u7387\u4f30\u8ba1\u3002\u8fd9\u65f6\u5019\uff0c\u6211\u4eec\u53ef\u4ee5\u7528\u4e4b\u524d\u63d0\u5230\u7684\u71b5\u6765\u8861\u91cf\u5206\u6b67\uff0c\u8fd9\u79cd\u65b9\u6cd5\u53eb\u505a\u6295\u7968\u71b5\uff08Vote Entropy\uff09\uff1a<\/p>\n<p>$$<br \/>\nu(x)=-\\sum_y \\frac{V(y)}{C} \\log \\frac{V(y)}{C}<br \/>\n$$<\/p>\n<p>\u5176\u4e2d $V(y)=\\left|\\left\\lbrace h_i \\mid h_i(x)=y \\right\\rbrace \\right|$ \u8868\u793a\u6295\u7968\u7ed9\u7c7b\u522b $y$ \u7684\u5206\u7c7b\u5668\u6570\u91cf\u3002<\/p>\n<pre><code class=\"language-python\">import numpy as np\nfrom sklearn.ensemble import VotingClassifier\nfrom sklearn.svm import LinearSVC, SVC\nfrom sklearn.tree import DecisionTreeClassifier\nfrom sklearn.linear_model import PassiveAggressiveClassifier\nfrom sklearn.naive_bayes import GaussianNB\nfrom sklearn.neighbors import KNeighborsClassifier\nfrom scipy.stats import entropy\nfrom matplotlib.pyplot import figure, subplot, title, contour, show\nfrom numpy import array, argmax, meshgrid, apply_along_axis, bincount, ravel, c_<\/code><\/pre>\n<pre><code class=\"language-python\"># \u8fd4\u56de\u96c6\u6210\u5206\u7c7b\u5668\u5bf9\u8f93\u5165\u6570\u636e X \u4e2d\u6bcf\u4e2a\u6837\u672c\u7684\u6295\u7968\u7ed3\u679c\ndef get_ensemble_votes(ensemble, X: np.ndarray, n_classes) -&gt; np.ndarray: \n    predictions = ensemble._predict(X)\n    return np.apply_along_axis(np.bincount, axis=1, arr=predictions, minlength=n_classes) #\u5bf9\u6bcf\u4e2a\u6837\u672c\u7684\u9884\u6d4b\u7ed3\u679c\u8fdb\u884c\u6295\u7968\u8ba1\u6570\uff0c\u786e\u4fdd\u6bcf\u4e2a\u7c7b\u522b\u7684\u6295\u7968\u6570\u90fd\u88ab\u7edf\u8ba1\u3002\n\n# \u8ba1\u7b97\u672a\u6807\u8bb0\u6837\u672c\u7684\u6295\u7968\u71b5\uff0c\u5e76\u8fd4\u56de\u71b5\u6700\u5927\u7684\u6837\u672c\u7d22\u5f15\u3002\ndef vote_entropy(ensemble: VotingClassifier, X: np.ndarray, train_idx: Sequence[int], n_classes: int) -&gt; int:\n    unknown_indices = array([i for i in range(X.shape[0]) if i not in train_idx])\n    votes = get_ensemble_votes(ensemble, X[unknown_indices], n_classes=n_classes)\n    utility = entropy(votes \/ len(ensemble.estimators), axis=1)\n    return unknown_indices[argmax(utility)]\n\ndef plot_ensemble_voting_decision_boundary(ensemble, X, y, train_idx):\n    xx, yy = meshgrid_from_bounds(X.min(0) - 1, X.max(0) + 1)\n\n    try:\n        proba = ensemble.predict_proba(np.c_[xx.ravel(), yy.ravel()])\n    except AttributeError:\n        votes = get_ensemble_votes(ensemble, np.c_[xx.ravel(), yy.ravel()], n_classes=len(np.unique(y)))\n        proba = (votes \/ len(ensemble.estimators))\n\n    contour(xx, yy, proba[:, 0].reshape(xx.shape), levels=[0.5])\n    plot_data(X, y, train_idx)<\/code><\/pre>\n<pre><code class=\"language-python\">train_idx = starting_samples[:]\n#  \u786c\u6295\u7968\nensemble = VotingClassifier([\n    (&#039;linear svm&#039;, LinearSVC(dual=False)),\n    (&#039;rbf svm&#039;, SVC()),\n    (&#039;poly svm&#039;, SVC(kernel=&#039;poly&#039;)),\n    (&#039;decision tree&#039;, DecisionTreeClassifier()),\n    (&#039;passive agressive&#039;, PassiveAggressiveClassifier()),\n    (&#039;naive bayes&#039;, GaussianNB()),\n    (&#039;knn&#039;, KNeighborsClassifier(n_neighbors=2))\n], voting=&#039;hard&#039;).fit(X[train_idx], y[train_idx])\n\nfigure(figsize=(15, 12))\nsubplot(3, 3, 1)\ntitle(f&#039;Initial Training Set&#039;)\nplot_ensemble_voting_decision_boundary(ensemble, X, y, train_idx)\nplot_data(X, y, train_idx)\n\nfor round in range(8):\n    new_sample = vote_entropy(ensemble, X, train_idx, n_classes=2)\n    train_idx.append(new_sample)\n\n    subplot(3, 3, round+2)\n    title(f&#039;Round {round + 1} of 8&#039;)\n    ensemble = ensemble.fit(X[train_idx], y[train_idx])\n    plot_ensemble_voting_decision_boundary(ensemble, X, y, train_idx)\n    plot_data(X, y, train_idx)\n\nshow()<\/code><\/pre>\n<p align=\"center\">\n  <img decoding=\"async\" src=\"https:\/\/gnnclub-1311496010.cos.ap-beijing.myqcloud.com\/wp-content\/uploads\/2025\/04\/20250419173836289.png\" style=\"height:700px\">\n<\/p>\n<h3>\u5171\u8bc6\u71b5<\/h3>\n<p>\u6700\u5927\u5316 $u(\\mathbf x) = -\\sum_y\\overline P(y|\\mathbf x)\\log \\overline P(y|\\mathbf x)$ \u7684\u6837\u672c\u67e5\u8be2\u6807\u7b7e\u3002<\/p>\n<p>$\\overline P(y|\\mathbf x) = \\frac{1}{C} \\sum_{h_i \\in \\mathcal C} \\hat P_i(y|\\mathbf x)$ \u662f\u7c7b $y$ \u7684\u5171\u8bc6\u6982\u7387\uff0c\u800c $\\hat P_i(y|\\mathbf x)$ \u662f\u96c6\u5408 $\\mathcal C$ \u4e2d\u5206\u7c7b\u5668 $h_i$ \u7684\u540e\u9a8c\u7c7b\u6982\u7387\u4f30\u8ba1\u3002<\/p>\n<pre><code class=\"language-python\">\ndef consensus_entropy(ensemble: VotingClassifier, X: np.ndarray, train_idx: Sequence[int]) -&gt; int:\n  unknown_indices = array([i for i in range(X.shape[0]) if i not in train_idx])\n  probas = ensemble.predict_proba(X[unknown_indices])\n  utility = entropy(probas, axis=1)\n  return unknown_indices[argmax(utility)]<\/code><\/pre>\n<pre><code class=\"language-python\">train_idx = starting_samples[:]\nensemble = VotingClassifier([\n  (&#039;logistic regression&#039;, LogisticRegression()),\n  (&#039;decision tree&#039;, DecisionTreeClassifier()),\n  (&#039;naive bayes&#039;, GaussianNB()),\n  (&#039;knn&#039;, KNeighborsClassifier(n_neighbors=2))\n], voting=&#039;soft&#039;).fit(X[train_idx], y[train_idx])\n\nfigure(figsize=(15, 12))\nsubplot(3, 3, 1)\ntitle(f&#039;Initial Training Set&#039;)\nplot_ensemble_voting_decision_boundary(ensemble, X, y, train_idx)\nplot_data(X, y, train_idx)\n\nfor round in range(8):\n  new_sample = consensus_entropy(ensemble, X, train_idx)\n  train_idx.append(new_sample)\n\n  subplot(3, 3, round+2)\n  title(f&#039;Round {round + 1} of 8&#039;)\n  ensemble = ensemble.fit(X[train_idx], y[train_idx])\n  plot_ensemble_voting_decision_boundary(ensemble, X, y, train_idx)\n  plot_data(X, y, train_idx)<\/code><\/pre>\n<p align=\"center\">\n  <img decoding=\"async\" src=\"https:\/\/gnnclub-1311496010.cos.ap-beijing.myqcloud.com\/wp-content\/uploads\/2025\/04\/20250419173924799.png\" style=\"height:700px\">\n<\/p>\n<h3>\u6700\u5927\u5206\u6b67<\/h3>\n<p>\u67e5\u8be2\u6837\u672c\u6807\u7b7e\uff0c\u4f7f $u(\\mathbf x) = \\sum_{h_i \\in \\mathcal C} D_{\\text{KL}}(\\hat P_i | \\overline P)$ \u6700\u5927\u5316\uff0c\u5373\u59d4\u5458\u4f1a\u6210\u5458\u4e0e\u5171\u8bc6\u4e4b\u95f4\u7684\u5e73\u5747 Kullback Leibler \u5206\u6b67\u6700\u5927\u3002<\/p>\n<pre><code class=\"language-python\">def max_disagreement(ensemble: VotingClassifier, X: np.ndarray, train_idx: Sequence[int]) -&gt; int:\n  unknown_indices = array([i for i in range(X.shape[0]) if i not in train_idx])\n\n  probas = ensemble._collect_probas(X[unknown_indices])\n  consensus = np.mean(probas, axis=0)\n  utility = array([\n       np.apply_along_axis(entropy, axis=1, arr=probas[:, i], qk=qk).sum()\n       for i, qk in enumerate(consensus)\n  ])\n\n  return unknown_indices[argmax(utility)]<\/code><\/pre>\n<pre><code class=\"language-python\">train_idx = starting_samples[:]\nensemble = VotingClassifier([\n  (&#039;logistic regression&#039;, LogisticRegression()),\n  (&#039;decision tree&#039;, DecisionTreeClassifier()),\n  (&#039;naive bayes&#039;, GaussianNB()),\n  (&#039;knn&#039;, KNeighborsClassifier(n_neighbors=2))\n], voting=&#039;soft&#039;).fit(X[train_idx], y[train_idx])\n\nfigure(figsize=(15, 12))\nsubplot(3, 3, 1)\ntitle(f&#039;Initial Training Set&#039;)\nplot_ensemble_voting_decision_boundary(ensemble, X, y, train_idx)\nplot_data(X, y, train_idx)\n\nfor round in range(8):\n  new_sample = max_disagreement(ensemble, X, train_idx)\n  train_idx.append(new_sample)\n\n  subplot(3, 3, round+2)\n  title(f&#039;Round {round + 1} of 8&#039;)\n  ensemble = ensemble.fit(X[train_idx], y[train_idx])\n  plot_ensemble_voting_decision_boundary(ensemble, X, y, train_idx)\n  plot_data(X, y, train_idx)<\/code><\/pre>\n<p align=\"center\">\n  <img decoding=\"async\" src=\"https:\/\/gnnclub-1311496010.cos.ap-beijing.myqcloud.com\/wp-content\/uploads\/2025\/04\/20250419174012722.png\" style=\"height:700px\">\n<\/p>\n<hr \/>\n<h2>\u9884\u671f\u6a21\u578b\u53d8\u5316\uff08Expected Model Change\uff09<\/h2>\n<ul>\n<li>\n<p>\u4e0d\u786e\u5b9a\u6027\u91c7\u6837\uff08Uncertainty Sampling\uff09 \u548c \u59d4\u5458\u4f1a\u67e5\u8be2\uff08QBC\u5728\u6982\u5ff5\u4e0a\u5f88\u76f8\u4f3c\uff1a\u4f60\u8bad\u7ec3\u4e00\u4e2a\u6a21\u578b\uff08\u6216\u4e00\u4e2a\u96c6\u6210\u6a21\u578b\uff09\uff0c\u7136\u540e\u7528\u5b83\u5bf9\u672a\u89c1\u8fc7\u6837\u672c\u7684\u8f93\u51fa\u6765\u5224\u65ad\u8fd9\u4e2a\u6837\u672c\u7684\u4ef7\u503c\u3002\u7136\u800c\uff0c\u8fd9\u79cd\u65b9\u6cd5\u53ef\u80fd\u4f1a\u9009\u62e9\u90a3\u4e9b\u9760\u8fd1\u51b3\u7b56\u8fb9\u754c\u7684\u6837\u672c\uff0c<strong>\u800c\u8fd9\u4e9b\u6837\u672c\u5e76\u4e0d\u4e00\u5b9a\u5bf9\u5b66\u4e60\u6570\u636e\u7684\u6574\u4f53\u7ed3\u6784\u6709\u5e2e\u52a9\u3002<\/strong><\/p>\n<\/li>\n<li>\n<p>\u9884\u671f\u6a21\u578b\u53d8\u5316\uff08EMC\uff09\u8bd5\u56fe\u89e3\u51b3\u8fd9\u4e2a\u95ee\u9898\uff0c\u5b83\u7684\u6838\u5fc3\u601d\u60f3\u662f\uff1a\u5173\u6ce8\u672a\u6765\u3002\u6211\u4eec\u4e0d\u518d\u95ee\u6a21\u578b\u5bf9\u67d0\u4e2a\u6837\u672c\u7684\u9884\u6d4b\u6709\u591a\u4e0d\u786e\u5b9a\uff0c\u800c\u662f\u95ee\uff1a\u5982\u679c\u6211\u4eec\u77e5\u9053\u8fd9\u4e2a\u6837\u672c\u7684\u6807\u7b7e\uff0c\u6a21\u578b\u4f1a\u53d1\u751f\u591a\u5927\u7684\u53d8\u5316\u3002\u90a3\u4e9b\u80fd\u8ba9\u6a21\u578b\u53d1\u751f\u6700\u5927\u53d8\u5316\u7684\u6837\u672c\uff0c\u5c31\u662f\u6700\u6709\u4ef7\u503c\u7684\u6837\u672c\u3002<\/p>\n<\/li>\n<\/ul>\n<h4><img decoding=\"async\" src=\"https:\/\/img.icons8.com\/?size=100&id=91CnU00i6HLv&format=png&color=000000\" style=\"height:50px;display:inline\"> \u5982\u4f55\u5b9e\u73b0\u9884\u671f\u6a21\u578b\u53d8\u5316\uff1f<\/h4>\n<p>\u6700\u76f4\u63a5\u7684\u65b9\u6cd5\u662f\uff1a\u4e3a\u6bcf\u4e2a\u53ef\u80fd\u7684\u6807\u7b7e\u8bad\u7ec3\u4e00\u4e2a\u6a21\u578b\uff0c\u7136\u540e\u6bd4\u8f83\u65b0\u65e7\u6a21\u578b\u4e4b\u95f4\u7684\u5dee\u5f02\u3002\u4e0d\u8fc7\uff0c\u8fd9\u79cd\u65b9\u6cd5\u8ba1\u7b97\u91cf\u975e\u5e38\u5927\uff0c\u901a\u5e38\u4e0d\u73b0\u5b9e\u3002\u5e78\u8fd0\u7684\u662f\uff0c\u6709\u4e00\u4e9b\u65b9\u6cd5\u53ef\u4ee5\u8ba9\u8fd9\u4e2a\u601d\u8def\u53d8\u5f97\u66f4\u53ef\u884c\u3002<\/p>\n<p>\u5bf9\u4e8e\u57fa\u4e8e\u68af\u5ea6\u7684\u4f18\u5316\u65b9\u6cd5\uff08\u6bd4\u5982\u6df1\u5ea6\u5b66\u4e60\uff09\uff0c\u6211\u4eec\u53ef\u4ee5\u7528<strong>\u8bad\u7ec3\u68af\u5ea6\u7684\u9884\u671f\u957f\u5ea6<\/strong>\u6765\u8fd1\u4f3c\u6a21\u578b\u7684\u53d8\u5316\u3002\u516c\u5f0f\u662f\uff1a<\/p>\n<p>$$<br \/>\nu(x)=\\mathbb{E}_y[|\\nabla \\ell(h ; \\ell \\cup(x, y))|] \\approx \\mathbb{E}_y[|\\nabla \\ell(h ;(x, y))|],<br \/>\n$$<\/p>\n<p>\u5176\u4e2d $\\ell(h ; \\ell)$ \u662f\u635f\u5931\u51fd\u6570\uff0c$\\ell$ \u662f\u8bad\u7ec3\u96c6\u3002\u8fd9\u4e2a\u8fd1\u4f3c\u57fa\u4e8e\u4e24\u4e2a\u5047\u8bbe\uff1a<\/p>\n<p>1\uff0e$h$ \u662f\u4e00\u4e2a\u5df2\u7ecf\u8bad\u7ec3\u597d\u7684\u6a21\u578b\u3002<\/p>\n<p>2\uff0e\u8bad\u7ec3\u6570\u636e\u662f\u72ec\u7acb\u540c\u5206\u5e03\u7684\u3002<\/p>\n<p>\u8fd9\u610f\u5473\u7740\u68af\u5ea6\u7684\u53d8\u5316\u53ea\u6765\u81ea\u4e8e\u6837\u672c $x$ \uff0c\u6240\u4ee5\u6211\u4eec\u53ea\u9700\u8981\u4e3a\u6bcf\u4e2a $x$ \u8ba1\u7b97\u635f\u5931\u51fd\u6570\uff0c\u800c\u4e0d\u9700\u8981\u4e3a\u6574\u4e2a\u8bad\u7ec3\u96c6\u8ba1\u7b97\u3002<\/p>\n<p>\u901a\u4fd7\u6765\u8bf4\uff0c\u9884\u671f\u6a21\u578b\u53d8\u5316\uff08EMC\uff09 \u7684\u6838\u5fc3\u601d\u60f3\u662f\uff1a<strong>\u9009\u62e9\u90a3\u4e9b\u80fd\u8ba9\u6a21\u578b\u53d1\u751f\u6700\u5927\u53d8\u5316\u7684\u6837\u672c<\/strong>\u3002\u4e0e\u4e0d\u786e\u5b9a\u6027\u91c7\u6837\u548c\u59d4\u5458\u4f1a\u67e5\u8be2\u4e0d\u540c\uff0cEMC \u5173\u6ce8\u7684\u662f\u6837\u672c\u5bf9\u6a21\u578b\u672a\u6765\u7684\u5f71\u54cd\uff0c\u800c\u4e0d\u662f\u6a21\u578b\u5f53\u524d\u7684\u9884\u6d4b\u4e0d\u786e\u5b9a\u6027\u3002<\/p>\n<pre><code class=\"language-python\">import torch\nimport torch.nn as nn\nimport torch.optim as optim\n\n# \u5047\u8bbe model \u662f\u4e00\u4e2a\u5df2\u7ecf\u8bad\u7ec3\u597d\u7684\u6a21\u578b\n# X_unlabeled \u662f\u672a\u6807\u8bb0\u7684\u6570\u636e\u96c6\n\ndef compute_gradients(model, X_unlabeled):\n    gradients = []\n    for x in X_unlabeled:\n        x = torch.tensor(x, requires_grad=True)\n        output = model(x)\n        loss = nn.CrossEntropyLoss()(output, model.predict(x))  # \u4f7f\u7528\u4f2a\u6807\u7b7e\n        loss.backward()\n        gradients.append(x.grad.norm())  # \u8ba1\u7b97\u68af\u5ea6\u7684\u8303\u6570\n    return gradients\n\ndef select_samples(model, X_unlabeled, top_k=10):\n    gradients = compute_gradients(model, X_unlabeled)\n    top_k_indices = torch.topk(torch.tensor(gradients), top_k).indices\n    return top_k_indices<\/code><\/pre>\n<hr \/>\n<h2>\u9884\u671f\u8bef\u5dee\u51cf\u5c11\uff08Expected Error Reduction, EER\uff09<\/h2>\n<p>\u65e8\u5728\u901a\u8fc7\u9009\u62e9\u90a3\u4e9b\u80fd\u591f\u6700\u5927\u7a0b\u5ea6\u51cf\u5c11\u6a21\u578b\u5728\u672a\u89c1\u8fc7\u6570\u636e\u4e0a\u7684\u9884\u6d4b\u8bef\u5dee\u7684\u6837\u672c\u8fdb\u884c\u6807\u6ce8\uff0c\u4ece\u800c\u63d0\u5347\u6a21\u578b\u7684\u6027\u80fd\u3002EER \u7684\u6838\u5fc3\u601d\u60f3\u662f\uff1a<strong>\u9009\u62e9\u90a3\u4e9b\u6807\u6ce8\u540e\u80fd\u591f\u663e\u8457\u51cf\u5c11\u6a21\u578b\u5728\u672a\u6765\u6570\u636e\u4e0a\u7684\u9884\u6d4b\u8bef\u5dee\u7684\u6837\u672c\u3002<\/strong><\/p>\n<p>EER \u7684\u6838\u5fc3\u95ee\u9898\u662f\uff1a<strong>\u54ea\u4e9b\u6837\u672c\u7684\u6807\u6ce8\u80fd\u591f\u6700\u5927\u7a0b\u5ea6\u5730\u51cf\u5c11\u6a21\u578b\u7684\u9884\u6d4b\u8bef\u5dee\uff1f<\/strong><\/p>\n<p>\u601d\u8def\uff1a<\/p>\n<ul>\n<li>\u5bf9\u4e8e\u6bcf\u4e2a\u672b\u6807\u8bb0\u6837\u672c $x$ \uff0c\u5047\u8bbe\u6211\u4eec\u77e5\u9053\u5b83\u7684\u771f\u5b9e\u6807\u7b7e $y$ \uff08\u4f2a\u6807\u7b7e\uff09\uff0c\u7136\u540e\u7528\u8fd9\u4e2a\u65b0\u6807\u6ce8\u7684\u6837\u672c $(x, y)$ \u6765\u66f4\u65b0\u6a21\u578b\u3002<\/li>\n<li>\u66f4\u65b0\u540e\u7684\u6a21\u578b\u4f1a\u5728\u5176\u4ed6\u672a\u6807\u8bb0\u6837\u672c $x^{\\prime}$ \u4e0a\u505a\u51fa\u66f4\u597d\u7684\u9884\u6d4b\uff0c\u4ece\u800c\u51cf\u5c11\u9884\u6d4b\u8bef\u5dee\u3002<\/li>\n<li>EER \u7684\u76ee\u6807\u662f\u9009\u62e9\u90a3\u4e9b\u80fd\u591f\u6700\u5927\u7a0b\u5ea6\u51cf\u5c11\u9884\u6d4b\u8bef\u5dee\u7684\u6837\u672c\u8fdb\u884c\u6807\u6ce8\u3002<\/li>\n<\/ul>\n<hr \/>\n<h2>\u5bc6\u5ea6\u52a0\u6743\uff08Density Weighting\uff09<\/h2>\n<p>\u5bc6\u5ea6\u52a0\u6743\u7684\u6838\u5fc3\u601d\u60f3\u662f\uff1a\u5728\u9009\u62e9\u6837\u672c\u8fdb\u884c\u6807\u6ce8\u65f6\uff0c\u4e0d\u4ec5\u8981\u8003\u8651\u6a21\u578b\u7684\u4e0d\u786e\u5b9a\u6027\uff0c\u8fd8\u8981\u8003\u8651\u6837\u672c\u5728\u6570\u636e\u5206\u5e03\u4e2d\u7684\u4ee3\u8868\u6027\u3002\u901a\u8fc7\u5c06\u6837\u672c\u7684\u6548\u7528\u4e0e\u5bc6\u5ea6\u4f30\u8ba1\u7ed3\u5408\u8d77\u6765\uff0c\u5bc6\u5ea6\u52a0\u6743\u80fd\u591f\u907f\u514d\u9009\u62e9\u90a3\u4e9b\u4f4d\u4e8e\u7a00\u758f\u533a\u57df\u6216\u5f02\u5e38\u503c\u7684\u6837\u672c\u3002<\/p>\n<p align=\"center\">\n  <img decoding=\"async\" src=\"https:\/\/gnnclub-1311496010.cos.ap-beijing.myqcloud.com\/wp-content\/uploads\/2025\/04\/20250419174055320.png\" style=\"height:300px\">\n<\/p>\n<p>\u5bc6\u5ea6\u52a0\u6743\u901a\u8fc7\u5c06\u6837\u672c\u7684\u6548\u7528 $u(x)$ \u4e0e\u6837\u672c\u5728\u7279\u5f81\u7a7a\u95f4\u4e2d\u7684\u5bc6\u5ea6\u4f30\u8ba1 $\\hat{p}(x)$ \u7ed3\u5408\u8d77\u6765\uff0c\u89e3\u51b3\u4e0a\u8ff0\u95ee\u9898\u3002\u5177\u4f53\u6765\u8bf4\uff0c\u5bc6\u5ea6\u52a0\u6743\u7684\u6548\u7528\u51fd\u6570\u5b9a\u4e49\u4e3a\uff1a<\/p>\n<p>$$<br \/>\nu^{\\prime}(x)=u(x) \\hat{p}(x)^\\beta<br \/>\n$$<\/p>\n<p>\u5176\u4e2d\uff1a<\/p>\n<p>\uff0d$u^{\\prime}(x)$ \u662f\u8c03\u6574\u540e\u7684\u6837\u672c\u6548\u7528\u3002<\/p>\n<p>\uff0d$\\hat{p}(x)$ \u662f\u6837\u672c $x$ \u7684\u5bc6\u5ea6\u4f30\u8ba1\u3002<\/p>\n<p>\uff0d$\\beta$ \u662f\u4e00\u4e2a\u8d85\u53c2\u6570\uff0c\u7528\u4e8e\u63a7\u5236\u5bc6\u5ea6\u52a0\u6743\u7684\u5f3a\u5ea6\u3002 <\/p>\n<p><strong>\u5bc6\u5ea6\u52a0\u6743\u7684\u4e3b\u8981\u4f5c\u7528\u662f<\/strong>\uff1a\u907f\u514d\u9009\u62e9\u7279\u5f81\u7a7a\u95f4\u4e2d\u7a00\u758f\u533a\u57df\u7684\u6837\u672c\u8fdb\u884c\u6807\u6ce8\u3002\u901a\u8fc7\u5f15\u5165\u5bc6\u5ea6\u4f30\u8ba1\uff0c\u5bc6\u5ea6\u52a0\u6743\u80fd\u591f\u786e\u4fdd\u9009\u62e9\u7684\u6837\u672c\u4e0d\u4ec5\u5177\u6709\u9ad8\u4e0d\u786e\u5b9a\u6027\uff0c\u800c\u4e14\u5728\u6570\u636e\u5206\u5e03\u4e2d\u5177\u6709\u4ee3\u8868\u6027\u3002\u8fd9\u6837\u53ef\u4ee5\u907f\u514d\u9009\u62e9\u5f02\u5e38\u503c\u6216\u79bb\u7fa4\u70b9\uff0c\u4ece\u800c\u63d0\u9ad8\u6807\u6ce8\u6570\u636e\u7684\u8d28\u91cf\u3002<\/p>\n<h4><img decoding=\"async\" src=\"https:\/\/img.icons8.com\/?size=100&id=91CnU00i6HLv&format=png&color=000000\" style=\"height:50px;display:inline\">  \u6211\u6709\u4e00\u4e2a\u95ee\u9898\uff0c\u4e3b\u52a8\u5b66\u4e60\u7684\u76ee\u7684\u4e0d\u662f\u4f7f\u5f97\u8bad\u7ec3\u66f4\u9ad8\u6548\u5417\uff1f\u53ef\u7b97\u5728\u5bfb\u627e\u4ef7\u503c\u6837\u672c\u7684\u65f6\u5019\u770b\u4e0a\u53bb\u4f1a\u6709\u5927\u91cf\u7684\u8ba1\u7b97\u6210\u672c\uff0c\u4f8b\u5982EER,EMC\u7b49\uff0c\u8fd9\u4e0d\u662f\u80cc\u9053\u800c\u9a70\u5417\uff1f\u8fd8\u662f\u8bf4\u4e3b\u52a8\u5b66\u4e60\u9664\u4e86\u9ad8\u6548\u8ba1\u7b97\uff0c\u8fd8\u6709\u5176\u4ed6\u76ee\u6807\uff0c\u4f8b\u5982\u8ba1\u7b97\u66f4\u51c6\uff1f<\/h4>\n","protected":false},"excerpt":{"rendered":"<p>Deep Learning create by Deepfinder Agenda \u5e08\u5f92\u76f8\u6388\uff1a\u6709\u76d1\u7763\u5b66\u4e60\uff08Su [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":3135,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[28],"tags":[],"class_list":["post-2948","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-28"],"_links":{"self":[{"href":"http:\/\/www.gnn.club\/index.php?rest_route=\/wp\/v2\/posts\/2948","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/www.gnn.club\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.gnn.club\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.gnn.club\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.gnn.club\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=2948"}],"version-history":[{"count":9,"href":"http:\/\/www.gnn.club\/index.php?rest_route=\/wp\/v2\/posts\/2948\/revisions"}],"predecessor-version":[{"id":2964,"href":"http:\/\/www.gnn.club\/index.php?rest_route=\/wp\/v2\/posts\/2948\/revisions\/2964"}],"wp:featuredmedia":[{"embeddable":true,"href":"http:\/\/www.gnn.club\/index.php?rest_route=\/wp\/v2\/media\/3135"}],"wp:attachment":[{"href":"http:\/\/www.gnn.club\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=2948"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.gnn.club\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=2948"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.gnn.club\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=2948"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}