$$y=\\frac{1}{1+e^{-z}}$$<\/span> \u7684\u51fd\u6570\u88ab\u79f0\u4e3a sigmod \u51fd\u6570\u3002<\/p>\n\u90a3\u4e48\u95ee\u9898\u6765\u4e86\uff1a\u8fd9\u4e24\u79cd\u56de\u5f52\u6a21\u578b\uff0c\u662f\u5426\u6709\u53ef\u80fd\u7edf\u4e00\u5230\u4e00\u79cd\u66f4\u52a0\u62bd\u8c61\u7684\u6a21\u578b\u4e2d\uff1f\u8fd9\u4e24\u79cd\u5f62\u5f0f\u5b8c\u5168\u4e0d\u4e00\u6837\u7684\u6a21\u578b\uff0c\u662f\u4e0d\u662f\u53ea\u662f\u67d0\u79cd\u66f4\u52a0\u5e7f\u4e49\u7684\u6a21\u578b\u7684\u7279\u4f8b\uff1f<\/p>\n
\u7b54\u6848\u662f\u80af\u5b9a\u7684\uff0c\u8fd9\u4e2a\u66f4\u52a0\u5e7f\u4e49\u3001\u62bd\u8c61\u7684\u6a21\u578b\u5c31\u53eb\u505a\u5e7f\u4e49\u7ebf\u6027\u6a21\u578b(Generalized Linear Model)\uff0c\u7b80\u79f0 GLM\u3002 \u5e38\u89c1\u7684\u7ebf\u6027\u56de\u5f52\u3001logistic \u56de\u5f52\u3001\u6cca\u677e\u56de\u5f52\u7b49\u7b49\u90fd\u662f\u5b83\u7684\u7279\u6b8a\u5f62\u5f0f\u3002<\/p>\n
\u4e0d\u4ec5\u5982\u6b64\uff0c\u5728 GLM \u7684\u89c6\u89d2\u4e0b\uff0c\u6211\u4eec\u80fd\u591f\u5408\u7406\u5730\u89e3\u91ca\uff1a\u4e3a\u4ec0\u4e48logistic \u56de\u5f52\u8981\u7528sigmod \u51fd\u6570 $y=\\frac{1}{1+e^{-z}}$<\/span>\uff1f\u4e3a\u4ec0\u4e48\u6cca\u677e\u56de\u5f52\u8981\u7528\u6307\u6570\u51fd\u6570$y=e^{z}$<\/span>\uff1f\u9009\u62e9\u8fd9\u4e9b\u51fd\u6570\u4e0d\u4ec5\u4ec5\u662f\u56e0\u4e3a\u4ed6\u4eec\u7684\u51fd\u6570\u66f2\u7ebf\u5f62\u72b6\u4e0e\u89c2\u6d4b\u503c\u5f88\u63a5\u8fd1\uff0c\u5176\u80cc\u540e\u6709\u66f4\u52a0\u6df1\u5c42\u6b21\u7684\u7edf\u8ba1\u57fa\u7840\u3002<\/p>\n1. \u5e7f\u4e49\u7ebf\u6027\u6a21\u578b\u7684\u5b9a\u4e49<\/h1>\n
\u5e7f\u4e49\u7ebf\u6027\u6a21\u578b\u5efa\u7acb\u5728\u4e09\u4e2a\u5b9a\u4e49\u7684\u57fa\u7840\u4e0a\uff0c\u5206\u522b\u4e3a\uff1a<\/p>\n
\n- \u5b9a\u4e49\u7ebf\u6027\u9884\u6d4b\u7b97\u5b50$$\\eta=\\theta^Tx$$<\/span><\/li>\n
- \u5b9a\u4e49y \u7684\u4f30\u8ba1\u503c $$h(x,\\theta)= E(y|x,\\theta)$$<\/span><\/li>\n
- \u5b9a\u4e49 y \u7684\u4f30\u503c\u6982\u7387\u5206\u5e03\u5c5e\u4e8e\u67d0\u79cd\u6307\u6570\u5206\u5e03\u65cf\uff1a$$\\Pr(y|x,\\theta)= b(y) exp({\\eta^TT(y)-a(\\eta)})$$<\/span><\/li>\n<\/ul>\n
\u63a5\u4e0b\u6765\u8be6\u7ec6\u89e3\u91ca\u5404\u4e2a\u5b9a\u4e49<\/p>\n
1.1. \u5b9a\u4e49\u4e00\uff1a\u7ebf\u6027\u9884\u6d4b\u7b97\u5b50<\/h2>\n
\u5e7f\u4e49\u7ebf\u6027\u6a21\u578b\u7684\u540d\u5b57\u4e2d\u6709\u300e\u7ebf\u6027\u300f\u4e24\u4e2a\u5b57\uff0c\u81ea\u7136\u5b83\u5305\u542b\u4e86\u7ebf\u6027\u8ba1\u7b97\u7684\u8fc7\u7a0b\uff0c\u4e5f\u5c31\u662f\u5b83\u7684\u5047\u8bbe\u4e4b\u4e00\uff0c\u5b9a\u4e49\u7ebf\u6027\u9884\u6d4b\u7b97\u5b50(linear predictor)\u4e3a\uff1a<\/p>\n
$$\\eta=\\theta^Tx$$<\/span><\/p>\n\u8fd9\u91cc\u7684\u00a0$\\theta$\u00a0<\/span>\u548c$x$<\/span>\u90fd\u662f\u5411\u91cf\uff0c\u5199\u6210\u6807\u91cf\u5f62\u5f0f\u5c31\u662f\uff1a<\/p>\n$$\\eta=\\theta_0x_0 + \\theta_1x_1 + \\cdots+ \\theta_nx_n$$<\/span><\/p>\n\u901a\u5e38 $x_0 = 1$<\/span>\u3002<\/p>\n\u4e0d\u8981\u95ee\u4e3a\u4f55\u8fd9\u4e48\u5b9a\u4e49\uff0c\u8fd9\u53ef\u4ee5\u7406\u89e3\u4e3a \u300e\u5e7f\u4e49\u7ebf\u6027\u6a21\u578b\u300f \u7ea6\u5b9a\u7684\u89c4\u5219\u3002<\/p>\n
1.2. \u5b9a\u4e49\u4e8c\uff1a\u671f\u671b\u4f30\u8ba1<\/h2>\n
\u5982\u679c\u4ee5\u6982\u7387\u8bba\u7684\u65b9\u5f0f\u89e3\u91ca\u56de\u5f52\uff08regression\uff09\u8fd9\u4e00\u8fc7\u7a0b\uff0c\u6211\u4eec\u53ef\u4ee5\u628a\u901a\u8fc7\u7ed9\u5b9a\u7684\u81ea\u53d8\u91cf $x$<\/span>\uff0c\u548c\u76f8\u5173\u7684\u7ebf\u6027\u53c2\u6570 $\\theta$<\/span>\u00a0\u4f30\u8ba1\u56e0\u53d8\u91cf y \u7684\u8fc7\u7a0b\u3002 \u7406\u89e3\u4e3a\u6c42\u89e3\u6761\u4ef6\u6982\u7387 $\\Pr(y|x,\\theta)$<\/span>\u00a0\u7684\u8fc7\u7a0b\u3002 \u4e5f\u5c31\u662f\u5728\u7ed9\u5b9a\u4e86$\\eta=\\theta^Tx$<\/span>\u7684\u6761\u4ef6\u4e0b\uff0c\u6c42\u89e3\u56e0\u53d8\u91cf y \u7684\u6982\u7387\u5206\u5e03\u66f2\u7ebf\u3002 \u7136\u540e\uff0c\u8ba1\u7b97\u8fd9\u4e2a\u6982\u7387\u5206\u5e03\u7684\u671f\u671b\u503c $E(y|x,\\theta)$<\/span>\uff0c\u4f5c\u4e3a y \u7684\u4f30\u8ba1\u503c\uff0c\u540c\u65f6\u8fd9\u4e2a\u6982\u7387\u5206\u5e03\u7684\u65b9\u5dee $Var(y|x,\\theta)$<\/span>\u4f5c\u4e3a y \u7684\u4f30\u8ba1\u503c\u7684\u65b9\u5dee\u3002<\/p>\n\u56e0\u6b64\u7b2c\u4e8c\u4e2a\u5047\u8bbe\u5c31\u662f\uff1ay \u7684\u4f30\u8ba1\u503c\u5c31\u662f $\\Pr(y|x,\\theta)$<\/span>\u00a0\u7684\u671f\u671b\u503c\u3002 \u5982\u679c\u7528 $h(x,\\theta)$<\/span>\u8868\u793a y \u7684\u4f30\u8ba1\u503c\uff0c\u8fd9\u4e00\u5047\u8bbe\u5199\u4e3a\uff1a<\/p>\n$$h(x,\\theta) = E(y|x,\\theta)$$<\/span><\/p>\n<\/p>\n
1.3. \u5b9a\u4e49\u4e09\uff1a\u6307\u6570\u5206\u5e03\u65cf<\/h2>\n
\u5728\u6beb\u65e0\u5934\u7eea\u7684\u60c5\u51b5\u4e0b\uff0c\u8981\u6c42\u89e3$\\Pr(y|x,\\theta)$<\/span>\u7684\u51fd\u6570\u8868\u8fbe\u5f0f\u4e0d\u592a\u53ef\u884c\u3002\u56e0\u6b64 \u5e7f\u4e49\u7ebf\u6027\u6a21\u578b\u505a\u51fa\u5047\u8bbe\uff0c$\\Pr(y|x)$<\/span>\u7684\u6982\u7387\u5206\u5e03\u670d\u4ece\u5982\u4e0b\u5f62\u5f0f\uff1a<\/p>\n$$\\Pr(y|x,\\theta) = b(y) e^{\\eta^TT(y)-a(\\eta)}$$<\/span><\/p>\n\u6ce8\u610f\uff0c\u8fd9\u91cc\u7684$\\eta=\\theta^Tx$<\/span>\uff0c\u56e0\u6b64\u8fd9\u4e2a\u6982\u7387\u5206\u5e03\u4e5f\u53ef\u4ee5\u5199\u6210\uff1a<\/p>\n$$\\Pr(y|\\eta) = b(y) e^{\\eta^TT(y)-a(\\eta)}$$<\/span><\/p>\n\u5176\u4e2d $b(y)$\uff0c$T(y)$ \u662f $y$ \u7684\u51fd\u6570\uff0c$a(\\eta)$<\/span>\u00a0\u662f $\\eta$<\/span>\u00a0\u7684\u51fd\u6570\u3002 a,b,T \u8fd9\u4e09\u4e2a\u51fd\u6570\u7684\u5f62\u5f0f\u672a\u77e5\uff0c\u662f\u4e00\u79cd\u62bd\u8c61\u7684\u8868\u8fbe\u65b9\u5f0f\u3002 \u4e00\u822c\u60c5\u51b5\u4e0b$T(y)=y$\uff0c\u540e\u6587\u4e2d\u6240\u6709 $T(y)$\u90fd\u4f1a\u76f4\u63a5\u5199\u6210$y$\u3002<\/p>\n\u51e1\u662f\u80fd\u5199\u6210\u8fd9\u4e2a\u5f62\u5f0f\u7684\u6982\u7387\u5206\u5e03\u51fd\u6570\uff0c\u90fd\u79f0\u4e4b\u4e3a\u6307\u6570\u5206\u5e03\u65cf\u4e2d\u7684\u4e00\u79cd\u7279\u4f8b\u3002 \u5982\u679c $a,b,T$ \u9009\u62e9\u67d0\u79cd\u7279\u6b8a\u7684\u51fd\u6570\u5f62\u5f0f\uff0c\u6307\u6570\u5206\u5e03\u65cf\u5c31\u4f1a\u9000\u5316\u4e3a\u67d0\u79cd\u7279\u6b8a\u7684\u6982\u7387\u5206\u5e03\uff08\u6bd4\u5982\u4e8c\u9879\u5206\u5e03\u3001\u6b63\u6001\u5206\u5e03\uff09\uff0c\u800c\u5177\u4f53\u7684\u5206\u5e03\u5f62\u5f0f\u5c31\u4f1a\u5bf9\u5e94\u4e86\u5177\u4f53\u7684\u56de\u5f52\u6a21\u578b\u3002<\/p>\n
\u540e\u6587\u5c06\u5177\u4f53\u4ecb\u7ecd a,b,T \u5728\u4ec0\u4e48\u5f62\u5f0f\u4e0b\u4f1a\u53d8\u4e3a\u5e38\u89c1\u7684\u56de\u5f52\u6a21\u578b\u3002\u672c\u6587\u8fd8\u5c06\u7ee7\u7eed\u5bf9\u8fd9\u4e2aGLM\u8fdb\u884c\u62bd\u8c61\u5730\u5206\u6790\u3002<\/p>\n
2. \u5e7f\u4e49\u7ebf\u6027\u6a21\u578b\u7684\u7279\u5f81<\/h1>\n
\u4e3a\u4ec0\u4e48\u8981\u628a y \u7684\u6761\u4ef6\u5206\u5e03\u5b9a\u4e49\u4e3a\u8fd9\u4e48\u5947\u602a\u7684\u6307\u6570\u5206\u5e03\u65cf\uff1f\u8fd9\u662f\u56e0\u4e3a\uff0c\u5728\u8fd9\u6837\u7684\u5b9a\u4e49\u4e0b\uff0c\u6211\u4eec\u53ef\u4ee5\u8bc1\u660e\uff1a<\/p>\n
\n- $\\Pr(y|\\eta)$<\/span>\u00a0\u7684\u671f\u671b\u503c $$E(y|\\eta)=\\frac{d}{d\\eta}a({\\eta})$$<\/span><\/li>\n
- $\\Pr(y|\\eta)$<\/span> \u7684\u65b9\u5dee $$Var(y|\\eta)=\\frac{d^2}{d\\eta^2}a({\\eta})$$<\/span><\/li>\n<\/ul>\n
\u5982\u6b64\u7b80\u6d01\u7684\u671f\u671b\u548c\u65b9\u5dee\u610f\u5473\u7740\uff1a\u4e00\u65e6\u5f85\u4f30\u8ba1\u7684 $y$ \u7684\u6982\u7387\u5206\u5e03\u5199\u6210\u4e86\u67d0\u79cd\u786e\u5b9a\u7684\u6307\u6570\u5206\u5e03\u65cf\u7684\u5f62\u5f0f\uff08\u4e5f\u5c31\u662f\u7ed9\u5b9a\u4e86\u5177\u4f53\u7684 a,b,T\uff09\uff0c\u90a3\u4e48\u6211\u4eec\u53ef\u4ee5\u76f4\u63a5\u5957\u7528\u516c\u5f0f $h(x,\\theta)=E(y|x,\\theta)=\\frac{d}{d\\eta}a({\\eta})$<\/span> \u6784\u5efa\u56de\u5f52\u6a21\u578b\u3002\u901a\u8fc7\u8fd9\u4e2a\u89c4\u5f8b\uff0c\u6211\u4eec\u53ef\u4ee5\u89e3\u91ca\u4e3a\u4ec0\u4e48 logistic \u56de\u5f52\u8981\u7528sigmod \u51fd\u6570 $y=\\frac{1}{1+e^{-z}}$<\/span> \u5efa\u6a21\u3002<\/p>\n\u6307\u6570\u5206\u5e03\u65cf\u7684\u8fd9\u4e24\u4e2a\u7279\u5f81\u5982\u6b64\u7b80\u6d01\uff0c\u4ee5\u81f3\u4e8e\u6211\u5fcd\u4e0d\u4f4f\u8bd5\u7740\u8bc1\u660e\u8fd9\u4e24\u4e2a\u7ed3\u8bba\u3002\u5982\u4f55\u901a\u8fc7 GLM \u5f97\u51fa\u5e38\u89c1\u7684\u7ebf\u6027\u6a21\u578b\u3001logistic \u56de\u5f52\u5219\u5728\u540e\u9762\u7684\u535a\u5ba2\u4e2d\u4ecb\u7ecd\u3002<\/p>\n
2.1. \u8bc1\u660e\u6307\u6570\u5206\u5e03\u65cf\u7684\u671f\u671b<\/h2>\n
\u9996\u5148\u5b9a\u4e49\u4f3c\u7136\u51fd\u6570 $L(y,\\eta)$<\/span>\uff1a<\/p>\n$$L(y,\\eta) = \\log \\Pr(y|\\eta) = \\log(b(y) e^{\\eta~y)-a(\\eta)})$$<\/span><\/p>\n\u5316\u7b80\u4e3a:<\/p>\n
$$L(y,\\eta) = \\log(b(y)) + \\eta~y – a(\\eta)$$<\/span><\/p>\n\u518d\u5b9a\u4e49\u5176\u5bf9 $\\eta$<\/span>\u7684\u5012\u6570 U:<\/p>\n$$U(y,\\eta) = \\frac{d}{d\\eta} L(y,\\eta) = y – \\frac{d}{d\\eta} a(\\eta)$$<\/span><\/p>\n\u53ef\u4ee5\u8bc1\u660e $U(y,\\eta)$<\/span> \u7684\u671f\u671b $E(U(y,\\eta))=0$<\/span>\uff08\u8bc1\u660e\u8fc7\u7a0b\u653e\u5728\u540e\u9762\uff09\uff0c\u90a3\u4e48\u6709 $$E(y – \\frac{d}{d\\eta} a(\\eta)) = 0$$<\/span> $$E(y)= E(\\frac{d}{d\\eta} a(\\eta))$$<\/span><\/p>\n\u56e0\u4e3a$$\\frac{d}{d\\eta} a(\\eta)$$<\/span>\u662f\u4e00\u4e2a\u4e0e y \u65e0\u5173\u7684\u51fd\u6570\uff0c\u56e0\u6b64$$\\frac{d}{d\\eta} a(\\eta)$$<\/span>\u7684\u671f\u671b\u5c31\u662f\u5b83\u672c\u8eab\uff0c\u56e0\u6b64<\/p>\n$$E(y)= \\frac{d}{d\\eta} a(\\eta)$$<\/span><\/p>\n2.1.1 \u8bc1\u660eE(U) = 0<\/h3>\n
\u63a5\u4e0b\u6765\u8bc1\u660e\u4e3a\u4ec0\u4e48 E(U) = 0<\/p>\n
\u4e4b\u524d\u5b9a\u4e49\u4e86 $$U(y,\\eta)=\\frac{d}{d\\eta}L(y,\\eta)=\\frac{d}{d\\eta}\\log\\Pr(y|\\eta)$$<\/span><\/p>\n\u5e26\u5165\u5176\u671f\u671b\u516c\u5f0f$$E(U(y,\\eta))=\\int{U(y,\\eta)\\Pr(y|\\eta)dy}$$<\/span>\u4e2d\uff0c\u5219\u6709\uff1a<\/p>\n$$E(U(y,\\eta))=\\int{\\frac{d}{d\\eta}\\log\\Pr(y|\\eta)\\Pr(y|\\eta)dy}$$<\/span><\/p>\n\u8003\u8651\u5230 $$d\\log\\Pr(y|\\eta) = \\frac{1}{\\Pr(y|\\eta)} d \\Pr(y|\\eta)$$<\/span>\uff0c\u6709<\/p>\n$$E(U(y,\\eta))=\\int{\\frac{1}{\\Pr(y|\\eta)}\\frac{d}{d\\eta}\\Pr(y|\\eta)\\Pr(y|\\eta)dy}$$<\/span><\/p>\n\u5316\u7b80\u5f97\u5230<\/p>\n
$$E(U(y,\\eta))=\\int{\\frac{d}{d\\eta}\\Pr(y|\\eta)dy}$$<\/span><\/p>\n\u5c06\u5fae\u5206\u7b26\u53f7\u63d0\u53d6\u51fa\u6765\uff0c\u6709<\/p>\n
$$E(U(y,\\eta))=\\frac{d}{d\\eta}\\int{\\Pr(y|\\eta)dy}$$<\/span><\/p>\n\u8003\u8651\u5230\u6982\u7387\u5206\u5e03\u7684\u5f52\u4e00\u5316\u6761\u4ef6$$\\int{\\Pr(y|\\eta)dy}=1$$<\/span>\uff0c\u5219\u6709<\/p>\n$$E(U(y,\\eta))=\\frac{d}{d\\eta}1$$<\/span><\/p>\n\u5e38\u6570\u7684\u5fae\u5206\u4e3a0\uff0c\u56e0\u6b64<\/p>\n
$$E(U(y,\\eta))=0$$<\/span><\/p>\n2.2. \u8bc1\u660e\u6307\u6570\u5206\u5e03\u65cf\u7684\u65b9\u5dee<\/h2>\n
\u5148\u5206\u6790$E(U^2)$<\/span>\uff0c\u5e26\u5165$U(y,\\eta) = y – \\frac{d}{d\\eta} a(\\eta)$<\/span>\uff0c\u6709\uff1a<\/p>\n$$E(U^2(y,\\eta))=\\int{(y – \\frac{d}{d\\eta} a(\\eta))^2\\Pr(y|\\eta)dy}$$<\/span><\/p>\n\u6ce8\u610f\uff0c\u521a\u521a\u5df2\u7ecf\u8bc1\u660e\u4e86$E(y)= \\frac{d}{d\\eta} a(\\eta)$<\/span>\uff0c\u5e26\u5165\u4e0a\u5f0f\u4e2d\uff0c\u6709<\/p>\n$$E(U^2(y,\\eta))=\\int{(y – E(y))^2\\Pr(y|\\eta)dy}$$<\/span><\/p>\n\u6ce8\u610f\uff0cy \u7684\u65b9\u5dee\u7684\u5b9a\u4e49\u5c31\u662f $Var(y|\\eta)=\\int{(y – E(y))^2\\Pr(y|\\eta)dy}$<\/span><\/p>\n\u6240\u4ee5\u6211\u4eec\u5f97\u5230\u7ed3\u8bba\uff1a<\/p>\n
$$Var(y|\\eta)=E(U^2(y,\\eta))$$<\/span><\/p>\n\u53c8\u56e0\u4e3a\u53ef\u4ee5\u8bc1\u660e $E(-\\frac{d}{d\\eta}U) = E(U^2)$<\/span>\uff08\u5177\u4f53\u8bc1\u660e\u8fc7\u7a0b\u653e\u5728\u6700\u540e\uff09\u3002 \u90a3\u4e48\u6709\uff1a<\/p>\n$$Var(y|\\eta)=E(-\\frac{d}{d\\eta}U)$$<\/span><\/p>\n\u73b0\u5728\u8981\u6c42\u89e3 $$E(-\\frac{d}{d\\eta}U(y,\\eta))$$<\/span>\uff0c\u5c06$$U(y,\\eta) = y – \\frac{d}{d\\eta} a(\\eta)$$<\/span>\u5e26\u5165\u5176\u4e2d\uff0c\u6709\uff1a<\/p>\n$$Var(y|\\eta)=E(-\\frac{d}{d\\eta}U(y,\\eta))=E(-\\frac{d}{d\\eta}(y – \\frac{d}{d\\eta} a(\\eta)))=E(\\frac{d^2}{d\\eta^2}a({\\eta}))$$<\/span><\/p>\n\u540c\u7406\uff0c\u5f0f\u5b50$$\\frac{d^2}{d\\eta^2}a({\\eta})$$<\/span>\u4e0e y \u65e0\u5173\uff0c\u5176\u671f\u671b\u4e3a\u5e38\u6570\uff0c\u5f97\u8bc1<\/p>\n$$Var(y|\\eta)=\\frac{d^2}{d\\eta^2}a({\\eta})$$<\/span><\/p>\n
![\"220px-Linear_regression.svg\"](\"http:\/\/xg1990.com\/blog\/wp-content\/uploads\/2015\/01\/220px-Linear_regression.svg_.png\")
<\/a><\/div>\n