当前位置: 首页  科学研究  学术交流  学术报告

计算机科学与技术学科机器学习与视觉研究所系列学术报告(林宙辰 北京大学)

发布者:付慧娟   发布时间:2021-12-05  浏览次数:121

浙江师范大学数学与计算机科学学院“机器学习与视觉”专题系列报告二



报告题目On Training Implicit Models

报告专家:林宙辰北京大学,杰青

报告时间2021年120609:50-10:30

报告地点:腾讯会议号699-492-742(浙师大MLV专题系列报告第三期)

报告摘要Implicit models have emerged as deep networks with infinite layers. They have good mathematical properties and can achieve competitive performance as traditional deep networks. However, their training is a big issue. Previous works employ the implicit differentiation and solve the exact gradient for the backward propagation. However, is it necessary to compute such an exact gradient (which is usually quite expensive) for training? To this end, we propose a novel gradient estimate for these implicit models, named phantom gradient, that 1) forgoes the costly approximation of the exact gradient; and 2) provides an update direction (empirically) preferable to the implicit model training. We theoretically analyze the condition under which a descent direction of the loss landscape could be found, and provide two specific instantiations of the phantom gradient based on unrolling and the Neumann series. Experiments on large-scale vision tasks demonstrate that these lightweight phantom gradients significantly accelerate the backward passes in training implicit models (roughly 1.7× speedup), and even boost the performance over approaches based on the exact gradient.

 

报告专家简介:

林宙辰北京大学

林宙辰,北京大学教授,CSIG/IAPR/IEEE Fellow,国家杰青,中国图象图形学学会机器视觉专委会主任,中国自动化学会模式识别与机器智能专委会副主任,中国计算机学会计算机视觉专委会常务委员,中国人工智能学会模式识别专委会常务委员。研究领域为机器学习、计算机视觉和数值优化。发表论文240余篇,谷歌引用2.2万余次,英文专著2本,中文专著1本,获2020年度中国计算机学会科学技术奖自然科学一等奖。多次担任CVPR ICCVNIPS/NeurIPSICMLIJCAIAAAIICLR领域主席,担任ICPR 2022程序共同主席、ICML 2022资深领域主席,曾任IEEE T-PAMI编委,现任IJCVOptimization Methods and Software编委。


邀请人: 郑忠龙