天天射综合网

与芝加哥大学计算机科学系合作举办

2019年秋季

TTIC 31020-机器学习入门(100学分)

授课教师: Kevin Kimpel

地点: TTIC 526B室

时间: 星期二和星期四-2:00-3:20pm

TTIC 31150-数学工具包(CMSC 31150)(100个单位)

授课教师: Madhur Tulsiani

地点: TTIC 526室

时间: 星期二和星期四-9:30-10:50am

TTIC 31080近似算法(CMSC 37503)(100个单位)

指导老师: 朱莉娅·楚佐伊

地点: TTIC 530室

时间: 星期一和星期三-1:30-2:50pm

TTIC 31000-TTIC的研究

地点: TTIC 526室

时间: 星期五

2020年冬季

TTIC 31050生物信息学与计算生物学导论

指导老师: 徐金波

地点: TTIC 530室

时间: 星期二和星期四-9:30-10:50am

TTIC 31230-深度学习基础(CMSC 31230)(100个单元)

讲师: David McAllester

地点: TTIC 526B室

时间: 星期二和星期四-2:00-3:20pm

TTIC 31010-算法(CMSC 37000)

讲师: Yury Makarychev

地点: TTIC 526室

时间: 星期二和星期四-9:30-10:50am

辅导会议:周三-3:00-4:00pm

TTIC 31070凸优化(CMSC 35470,BUSN 36903,STAT 31015,CAAM 31015)

讲师: Nati Srebro

地点: TTIC 526B室

时间: 星期一和星期三-1:30-2:50pm

TTIC 31000-TTIC的研究

地点: TTIC 526室

时间: 星期五

2020年春季

TTIC 31180-概率图形模型(100个单位)

授课教师: Matthew Walter

地点: TTIC 530室

时间: 星期二和星期四-9:30-10:50am

TTIC 31110-语音技术(100个单位)

授课教师: Karen Livescu

地点: TTIC 526室

时间: 星期二和星期四-2:00-3:20pm

TTIC 31040-计算机视觉入门(100个单元)

授课老师: Greg Shakhnarovich

地点: TTIC 526室

时间: 星期一和星期三-1:30-2:50pm

TTIC 31250机器学习理论导论

授课教师: Avrim Blum

地点: TTIC 526室

时间: 星期一和星期三-3:00-4:20pm

TTIC 31000-TTIC的研究

地点: TTIC 526室

时间: 星期五


完整课程清单

TTIC 31000-TTIC的研究

网络表格

100单位

待定

TTIC研究人员每周进行的讲座和讨论,介绍他们的研究和研究问题。 提供TTIC进行的广泛研究。 课程为通过/失败学分。 满足四分之一的学分(需要三分学分)才能满足“ TTIC系列研究”的要求。 (有关详情,请参见学术课程指南)

TTIC 31010-算法(CMSC 37000)

100单位

尤里马卡里切夫

这是一门有关算法的研究生课程,重点是中央组合优化问题以及算法设计和分析的高级方法。 涵盖的主题包括贪婪算法,动态规划,随机算法和概率方法,组合优化和逼近算法,线性规划和在线算法。

该课程的教科书是Kleinberg和Tardos的“算法设计”

演讲计划
  1. 贪婪算法(1周)
  2. 动态编程(1周)
  3. 在线算法(1周)4-6最大流量,最小切割,二分匹配及其应用(3周)
  4. 线性编程,LP对偶(1周)
  5. NP硬度(1周)
  6. 近似算法(1周)
  7. 随机算法(1周)

假设熟悉证明和渐近符号。 还需要有关NP硬度概念的一些基本知识。

预期成果:

  • 能够使用贪婪或动态编程等范例设计和严格分析算法。
  • 了解优化中线性编程的用法。 能够将问题表达为线性程序。
  • 了解线性编程的对偶性,以及对最大流量/最小切割等问题的应用。 能够为线性程序编写对偶。

先决条件:假定熟悉证明和渐近符号。 还需要有关NP硬度概念的一些基本知识。

TTIC 31020-机器学习简介

100单位

凯文·金珀

机器学习的系统介绍,涵盖使用统计方法的理论和实践方面。 主题包括用于分类和回归的线性模型,支持向量机,正则化和模型选择,以及结构化预测和深度学习简介。 应用示例来自诸如信息检索,自然语言处理,计算机视觉等领域。

先修课程:概率,线性代数,本科生算法。

演讲计划

我将专注于有监督的学习,并且仅在必要时谈论无监督的设置(例如,混合模型和分类生成方法的密度估计)。 因此,没有群集。 将有20个1.5小时的讲座; 括号中的数字是每个主题的估计讲座数。

  • 机器学习,动机等简介(1)
  • 关于概率和代数的复习(1)
  • 统计学习框架; 损失/风险; 最小二乘回归(1)
  • 噪声模型; 错误分解; 偏差/方差和过度拟合(1)
  • 估计理论 ML / MAP,过度拟合,偏差/方差(1)
  • 模型复杂度,回归中的稀疏性(L1 / L2); L0稀疏性的逐步方法(1)
  • 分类; Fisher的LDA,逻辑回归和softmax(1)
  • 合奏方法,增强方法(1)
  • 生成模型,朴素贝叶斯,多元高斯(1)
  • 混合模型 电磁(2)
  • SVM和内核(2)
  • 非参数方法 最近邻,密度模拟(1)
  • 多层神经网络和深度学习(1)
  • 信息理论与学习; 信息标准,MDL及其与正则化的联系(1)
  • ML中的实验设计和评估(1)
  • 高级主题待定(1)
  • 总结和审查(1)

先决条件:基本线性代数,概率和微积分的知识。

预期成果:

  • 了解将模型拟合到数据和概念的概念,例如模型复杂性,过度拟合和泛化以及估计中的偏差方差折衷。
  • 学习并能够应用一些基本的学习方法,例如逻辑回归,支持向量机,提升,决策树,神经网络。
  • 了解优化技术的基础知识,例如梯度下降和通用EM算法。
  • 熟悉多元高斯和高斯混合。
  • 了解信息论中的基本概念(熵,KL散度)及其与机器学习的关系。

TTIC 31030-数学基础

100单位

戴维·麦克阿莱斯特

本课程从经典(非建设性)类型理论的角度涵盖数学基础,数学结构的一般概念,同构的一般概念以及公理化和构造在数学定义中的作用。 实数的定义用作基本示例。 本课程还涵盖了类型良好的形式主义中的可定义性概念。 一个主要的例子是向量空间与其对偶之间的线性双射的不可定义性。 强调了与机器学习相关的本体(类型),例如PCA,CCA和Banach空间(规范和对偶规范)的类型理论。

演讲计划

每周又有两次讲座。 简介和课程大纲。

第一部分:具有抽象障碍的逻辑。

后续推理规则和证明。 类型和变量声明。 自由和绑定变量。 结构和同构。

同构在抽象界面下等效。

第二部分:抽象案例研究

自然数,整数,有理数和实数。 向量空间。 对不存在规范基础(坐标系),规范内积或对偶空间的规范同构的不存在的形式处理。 矩阵代数的无坐标处理。

矩阵(运算符)类型之间的等效项。 最小二乘回归不需要环境内积,而正则化则需要。 梯度下降。 梯度下降需要内部环境积。 牛顿的方法没有。 向量空间上的概率分布的协方差矩阵。 多元中心极限定理的事实不需要环境内积。 典型的相关分析也不需要环境内部产物。 PCA需要和环境内部产品。

规范和Banach空间。 双重规范。 测量空间。 希尔伯特空间。 可微流形。 信息几何。 杰弗里(Jeffery)的先前。 自然梯度下降。

预期成果:

  • 能够就各种数学概念正式写出严格的证明和推理。
  • 从算子的角度了解线性代数,特征值和特征向量的基础。
  • 从操作员的角度了解各种算法,例如SVM,PCA,CCA和梯度下降。

TTIC 31040-计算机视觉简介(CMSC 35040)

100单位

格雷戈里Shakhnarovich

介绍计算机视觉的原理和实践。 本课程将对现代视觉主要任务中涉及的主题进行深入的调查,并提供一些实践经验。

先决条件:必须具有线性代数和概率(本科水平)的良好背景,以及机器学习(TTIC 31020或同等学历)的背景。 建议使用CMSC 25040。

话题:

  • 图像形成,表示和压缩
  • 颜色的表示和感知
  • 滤波和边缘检测
  • 图像特征,检测器和兴趣点算子
  • 模型拟合,RANSAC和Hough变换
  • 立体和多视图几何
  • 相机校准
  • 运动的表现与感知
  • 边缘和区域的表示和建模
  • 语义视觉:识别,检测和相关任务

预期成果:

  • 熟悉图像形成和图像分析模型。
  • 熟悉图像处理,对齐和匹配的主要方法。
  • 从图像和视频中进行3D重建的原理和方法的知识,以及构建和诊断这些方法的实现的能力。
  • 掌握了从图像对对象和场景进行分类的现代方法,并具有构建和诊断此类方法的实现的能力。

TTIC 31050-生物信息学和计算生物学导论

100单位

徐金波

演讲计划

本课程将侧重于数学模型和计算机算法在分子生物学研究中的应用。 特别是,本课程将涵盖以下主题。

  1. 同源搜索(1周)
  2. 序列比对和基序发现(1周)
  3. 下一代测序和基因组组装(1周)
  4. 蛋白质序列/结构分析,包括比对,分类,结构和功能预测(2周)。
  5. RNA序列/结构分析,包括比对,分类和预测(1周)
  6. 基因表达分析(1周)
  7. 生物网络分析(1周)
  8. 系统发育(1周)

预期成果:

  • 使用流行的生物信息学工具产生生物学上有意义的结果的能力
  • 能够解释由生物信息学工具产生的生物学结果
  • 能够针对重要的生物学问题实施基本的机器学习和优化算法
  • 支持向量机,隐马尔可夫模型,动态规划等基本模型和算法在解决生物学问题中的应用

先决条件:无

TTIC 31060-可计算性和复杂性理论(CMSC 38500,MATH 30500)

100单位

亚历山大·拉兹伯罗夫

第一部分由定义可计算函数的模型组成:原始递归函数,(通用)递归函数和Turing机器; 教堂转向论文; 无法解决的问题; 对角化 和可计算枚举集的性质。 第二部分涉及Kolmogorov(资源有限)的复杂性:单个对象中的信息量。 第三部分介绍了可与图灵机的时间和空间范围一起计算的函数:多项式时间可计算性,P和NP类,NP-完全问题,多项式时间层次和P-空间完全问题。

演讲计划

该课程的暂定计划是执行以下三个部分。

第一部分:用于定义可计算函数的模型:原始递归函数,(通用)递归函数和Turing机器; 教堂转向论文; 无法解决的问题; 对角化 和可计算枚举集的性质。

第二部分 Kolmogorov(资源有限)复杂性:单个对象中的信息量。

第三部分 可与图灵机的时间和空间范围一起计算的函数:多项式时间可计算性,P和NP类,NP完全问题,多项式时间层次和P空间完全问题。

这是该课程以前版本的网页: http : //people.cs.uchicago.edu/~razborov/teaching/spring14.html 。 但是我将非常愿意根据参加该课程的学生的背景来调整速度,样式和内容。

预期成果:

  • 识别各种情况下出现的算法问题的能力。
  • 将经典递归理论,Kolmogorov复杂度和基本计算复杂度应用到分析数学及其以外的问题中。
  • 识别并识别可解决和不可解决的问题,具有高/低Kolmogorov复杂度的问题以及基本复杂度类别的已完成问题。

TTIC 31070-凸优化(CMSC 35470,BUSN 36903,STAT 31015,CAAM 31015)

100单位

纳蒂斯雷布罗

该课程将涵盖无约束和约束凸优化的技术以及凸对偶性的实用介绍。 本课程的重点是(1)制定和理解凸优化问题并研究其性质; (2)理解和运用双重性; (3)提出和理解优化方法,包括用于非光滑问题的内点方法和一阶方法。 示例将主要来自数据拟合,统计和机器学习。

先修课程:线性代数,多维微积分,本科生算法

具体主题:

  • 优化问题的形式化
  • 平滑无约束的优化:梯度下降,共轭梯度下降,牛顿和拟牛顿法; 线搜索方法
  • 约束优化的标准公式:线性,二次,圆锥和半定规划
  • KKT最优条件
  • 拉格朗日对偶,约束条件,弱对偶和强对偶
  • 芬切尔共轭及其与拉格朗日对偶的关系
  • 多目标优化
  • 等式约束牛顿法
  • 对数屏障(中心路径)方法和基本对偶优化方法
  • 作为活动集方法示例的单纯形方法概述。
  • 一阶预言模型,子梯度和分离预言以及Ellipsoid算法的概述。
  • 大规模次梯度方法:次梯度下降,镜像下降,随机次梯度下降。

预期成果:

  • 能够根据所需的信息/访问,假设,迭代复杂度和运行时间来讨论和理解优化问题
  • 识别凸和非凸优化问题的能力
  • 能够对优化算法的选择做出明智的选择
  • 能够导出对偶问题,并使用对偶和KKT条件推理出最优解
  • 熟悉无约束的优化方法,包括梯度下降,共轭梯度下降,牛顿法和拟牛顿法
  • 熟悉用于约束优化的内部点方法
  • 熟悉包括线性规划,二次规划和半定规划在内的标准公式
  • 能够将抽象问题转换为受约束的优化问题,并且可以按照标准制定

TTIC 31080-近似算法(CMSC 37503)

100单位

朱卓娅·楚佐伊

这是一门有关逼近算法的基础课程,主要侧重于解决中央组合优化问题的逼近算法。 我们将主要关注经典算法的结果,但还将介绍一些最新的结果以及在近似方面的挑战。 该课程将涵盖主要的算法技术,包括LP舍入,原始对偶模式,度量方法,SDP舍入等。 虽然本课程的主要重点是算法,但我们还将讨论算法设计和下界证明之间的近似以及联系的下界。

假设掌握算法课程中涵盖的材料知识。

预期成果:

  • 了解诸如逼近因子,多项式时间逼近方案和逼近硬度的概念。
  • 了解线性程序(LP)在近似算法设计中的应用。 学习分析LP的舍入算法并了解完整性差距。 能够应用LP对偶。
  • 了解半定规划及其在近似中的应用。

先决条件:算法(TTIC31010或CMSC 37000)

TTIC 31090-信号,系统和随机过程

100单位

卡伦,利文斯库

研究生水平的信号和线性时不变系统分析入门。 主题包括:连续时间和离散时间变换(Fourier等); 线性滤波 采样和混叠; 随机过程及其与线性系统的相互作用; 在语音和图像处理以及机器人技术领域的应用。

先决条件:熟悉基本线性代数,特征值和特征向量的概念以及(多元)高斯分布。

预期成果:

  • 了解线性时不变(LTI)系统的特性和本征函数。
  • 了解傅立叶级数(离散时间和连续时间),傅立叶变换和卷积。
  • 能够分析随机过程,了解其平稳性及其与LTI系统的相互作用。
  • 了解有关采样和混叠,最小化均方误差的信号估计以及随机过程的参数估计的信息。 了解Karhunen-Loeve变换。

TTIC 31100-计算几何和公制几何(CMSC 39010)

100单位

尤里马卡里切夫

该课程涵盖计算和度量几何的基本概念,算法和技术。 涵盖的主题包括:凸包,多边形三角剖分,范围搜索,线段相交,Voronoi图,Delaunay三角剖分,度量和范数空间,低失真度量嵌入及其在近似算法中的应用,度量空间的填充分解,Johnson-Lindenstrauss变换和降维,近似最近邻搜索和对位置敏感的哈希。

该课程的教科书是M. de Berg,O。Cheong,M。van Kreveld,M。Overmars编写的“计算几何”。

演讲计划:
  1. 凸性:凸集,凸包,顶点,辅助线,边,不同的定义和基本特性,Caratheodory定理
  2. 凸包和线段相交:Jarvis March,Andrew算法,扫掠线算法,线段相交,Bentley-Ottmann算法
  3. 平面图和覆盖图:图形,图形图,平面图和平面图,欧拉公式,平面图的数据结构,计算覆盖图
  4. 正交范围搜索(2个讲座):二分搜索,kd树,范围树
  5. 点位置:梯形图,随机算法
  6. Voronoi图:Voronoi图,Fortune算法
  7. Delaunay三角剖分(1.5个讲座):三角剖分,Delaunay和局部Delaunay三角剖分:定义,存在与等价,Delaunay三角剖分与Voronoi图之间的对偶性,角度最优性
  8. 度量空间,赋范空间,低失真度量嵌入(1.5个讲座):度量空间和赋范空间,Lipschitz映射,失真,嵌入到Lp和lp中的嵌入
  9. 布尔金定理
  10. 稀疏切割:稀疏切割的近似算法
  11. 最小平衡割,最小线性排列,具有非均匀需求的最稀割,扩展器:用于平衡割和最小线性排列的polylog近似算法,扩展图,稀疏割的完整性缺口,具有非均匀需求的最稀割
  12. 最小多路切割,最小多路切割:最小多路切割和最小多路切割的近似算法
  13. 填充分解,分层分离的树,应用程序(2个讲座)
  14. 半定规划,Arora,Rao和Vazirani算法:半定规划,ARV(高级概述),增量分隔集,匹配封面
  15. 降维,最近邻居搜索:降维,近似最近邻居搜索,位置敏感哈希
  16. 局部敏感哈希,p稳定随机变量:局部敏感哈希,p稳定随机变量

预期成果:

  • 了解解决几何问题的标准算法和数据结构
  • 能够设计有效的算法和数据结构来解决几何问题
  • 了解度量几何的基本概念,例如度量和规范空间,低失真嵌入,降维,最近邻居搜索。
  • 了解度量几何在近似算法和计算机科学其他领域的应用。

先决条件:本科水平的算法,线性代数和概率分类; 具有良好的数学分析/微积分背景

TTIC 31110-语音技术

100单位

卡伦,利文斯库

本课程将介绍语音技术中使用的技术,主要侧重于语音识别和相关任务。 语音识别是最古老,最复杂的结构化序列预测任务之一,受到了广泛的研究和商业关注,因此,对于在涉及序列建模的人工智能其他领域中使用的许多技术提供了很好的案例研究。 该课程将详细介绍核心技术,包括隐马尔可夫模型,递归神经网络和条件随机场。 该课程将包括实际的家庭作业练习,我们将在其中构建和试验语音处理模型。 最后,它将包括语音与其他形式(如图像和文本)之间的连接的采样。

先决条件:具有基本概率的良好背景,最好是机器学习入门。

预期成果:

  • 了解并应用用于分析语音时间序列的工具,例如傅立叶分析和动态时间扭曲。
  • 了解并应用隐藏的马尔可夫模型,高斯混合以及语音问题的EM算法。
  • 了解并应用语言模型,平滑技术及其在语音识别中的应用。
  • 了解针对语音问题的生成性和区分性结构化预测方法。
  • 了解并应用现代深度学习工具执行语音处理任务。

TTIC 31120-统计与计算学习理论

100单位

纳蒂斯雷布罗

我们将讨论统计学习理论的经典成果和最新进展(主要是在不可知论的PAC模型下),探讨计算学习理论,并探讨与随机优化和在线遗憾分析的关系。 我们的重点将放在概念发展和对机器学习的严格定量理解上。 我们还将研究用于分析和证明学习方法的性能保证的技术。

先决条件:数学上的成熟度,例如在严格的分析过程中获得的。 离散数学(特别是组合和渐近符号)概率论机器学习算法简介; 基本复杂性理论(NP-Hardness)熟悉凸优化,计算复杂性和统计学背景可能会有所帮助,但不是必需的。

具体主题:

  • 统计模型(基于IID样本学习):
    • PAC(大概近似正确)和不可知论的PAC模型。
    • 随机优化
    • 基数界限
    • 说明长度界限
    • PAC-贝叶斯
    • 压缩界限
    • 增长函数和VC维
    • VC子图维和胖碎维
    • VC和胖碎维度对学习的严格表征
    • 覆盖号码
    • Rademacher平均值,包括本地Rademacher分析
  • 统一学习和免费午餐定理
  • 在线学习,在线优化和在线遗憾
    • 感知器规则和在线梯度下降
    • 专家与Winnow规则
    • Bregman发散和在线镜像下降
    • 在线到批量转换
  • 计算下界:
    • 正确学习的计算难度
    • 学习的密码学难度
  • 其他主题
    • 基于稳定性的分析
    • 提升:学习不足和提升的边际解释。

预期成果:

  • 能够识别不同的学习模型并对学习方法做出严格的陈述
  • 使用标准技术证明学习保证的能力
  • 证明学习问题的下限的能力

课程网页

TTIC 31140-图形模型的学习和推断

100单位

乌尔塔孙拉奎尔

图形模型是一种概率模型,其中随机变量之间的条件相关性通过图形指定。 图形模型提供了一个灵活的框架,可以为具有复杂交互作用的大量变量建模,这一点已得到广泛应用的证明,例如机器学习,计算机视觉,语音和计算生物学。 本课程将对图形模型中的学习和推理方法进行全面的调查,包括变异方法,原始对偶方法和采样技术。

TTIC 31150-数学工具包(CMSC 31150)

100单位

Ohannessian,Mesrob

该课程针对一年级研究生和高级本科生。 本课程的目的是收集并介绍在计算机科学的不同领域中使用的重要数学工具。 该课程将主要侧重于线性代数和概率。

我们打算涵盖以下主题和示例:

  • 抽象线性代数:向量空间,线性变换,希尔伯特空间,内积,Gram-Schmidt正交化,特征值和特征向量,SVD,最小二乘(约束不足/过度约束)
  • 离散概率:随机变量,马尔可夫,切比雪夫和切尔诺夫边界。
  • 高斯变量,浓度不等式,降维
  • 马丁格莱斯(时间允许)
  • 随机过程(时间允许)

预期成果:

  • 能够编写正确键入的严格证明。
  • 在抽象向量空间的背景下理解线性代数的各种概念。
  • 能够理解和分析随机过程。 熟悉离散和连续的随机变量以及各种浓度范围。

先决条件:无

TTIC 31160-生物信息学主题

100单位

徐金波

TTIC 31160将专注于数学模型和计算机算法在研究结构生物学,特别是蛋白质,RNA和DNA分子结构方面的应用。

这是我将在本课程中涵盖的主题列表。

  • 分子结构概论(1周)
  • 用于生物序列分析的生物信息学(1周)
  • 分子结构比较和比对的算法(1周)
  • 蛋白质二级结构预测算法(0.5周)
  • 蛋白质三级结构预测算法(1周)
  • RNA二级结构预测算法(1周)
  • RNA三级结构预测算法(0.5周)
  • 蛋白质对接算法(1周)
  • 蛋白质-蛋白质和蛋白质-RNA相互作用预测的算法(1周)
  • 确定染色质结构的算法(1周)

将会有家庭作业和最终项目。

预期成果:

  • 能够将结构生物学问题表达为数学问题
  • 应用高级优化算法(线性规划,半定规划和图形算法)和机器学习模型(概率图形模型)来解决结构生物信息学中的重要问题
  • 精通当前结构生物信息学的热门话题
  • 能够进行结构生物信息学的半独立研究

先决条件:无

TTIC 31170-机器人与人工智能的计划,学习和评估

100单位

沃尔特(Matthew)

本课程涉及机器人技术和人工智能(AI)的基本技术,重点在于不确定性下的概率推理,学习和计划。 本课程将研究作为严格数学工具的这些主题的理论基础,这些数学工具可以解决从机器人技术和AI广泛吸收的现实问题。 该课程将涵盖以下主题:贝叶斯滤波(卡尔曼滤波,粒子滤波和动态贝叶斯网络),同时定位和制图,计划,马尔可夫决策过程,部分可观察的马尔可夫决策过程,强化学习和图形模型。

预期成果:

  • 了解动态系统中建模和缓解不确定性的作用和概率技术
  • 证明有能力得出递归贝叶斯估计的分析和非参数解
  • 制定表示机器人定位和制图问题的概率模型,并展示这些模型如何提供递归估计中的技术
  • 了解确定性和随机域内的计划/搜索和决策算法
  • 演示实现用于减轻不确定性的最新算法并将这些技术应用于新问题和新领域的能力

先决条件:基本熟悉线性代数; 概率论背景; 基本的编程经验

TTIC 31180-概率图形模型

100单位

沃尔特(Matthew)

机器学习,计算机视觉,自然语言处理,机器人技术,计算生物学等领域的许多问题都需要对大型,异构变量集合之间的复杂交互进行建模。 图形模型结合了概率论和图论,以提供一个统一的框架,以紧凑,结构化的形式表示这些关系。 概率图形模型通过图将多变量联合分布分解为随机变量的小子集之间的一组局部关系。 这些局部交互导致条件独立,从而提供有效的学习和推理算法。 此外,它们的模块化结构提供了用于表达特定领域知识的直观语言,并有助于将建模进展转移到新应用程序中。

天天射综合网 This graduate-level course will provide a strong foundation for learning and inference with probabilistic graphical models. The course will first introduce the underlying representational power of graphical models, including Bayesian and Markov networks, and dynamic Bayesian networks. Next, the course will investigate contemporary approaches to statistical inference, both exact and approximate. The course will then survey state-of-the-art methods for learning the structure and parameters of graphical models.

Lecture Plan:
  • Review of probability theory
  • Directed graphical models
  • Undirected graphical models
  • Conditional random fields
  • Temporal models (hidden Markov models)
  • Exponential families
  • Variable elimination, sum-product
  • Belief propagation
  • MAP inference
  • Variational inference
  • Sampling-based inference
  • Dual decomposition
  • Multivariate Gaussians: Kalman filters and smoothing
  • Learning for directed graphical models
  • Learning for undirected graphical models
  • Learning given unobserved data
  • Learning for Gaussian models

天天射综合网 Expected outcomes:

  • Understand the representation of graphical models, including directed, undirected, and factor graph representations; factorization and Markov properties; and common spatial, temporal, hierarchical, and relational models.
  • Develop a solid understanding of exponential families and the related issues of conjugate priors, ML estimation, and parameter estimation in directed and undirected graphical models.
  • Demonstrate a familiarity with Gaussian graphical models, including Bayesian networks, Markov random fields, and inference algorithms under these models.
  • Understand methods for exact inference, including variable elimination, belief propagation (message passing), and the junction tree algorithm.
  • Understand techniques for approximate inference, including variational and Monte Carlo methods (eg, Gibbs sampling, Rao-Blackwellization, and Metropolis-Hastings).
  • Understand techniques for learning the structure and parameters of different families of graphical models both from observed and latent data.

Prerequisites: TTIC 31020 (or equivalent)

TTIC 31190 - Natural Language Processing

天天射综合网 100 units

天天射综合网 Gimpel, Kevin

天天射综合网 This course will introduce fundamental concepts in natural language processing (NLP). NLP includes a range of research problems that involve computing with natural language. Some are user-facing applications, such as spam classification, question answering, summarization, and machine translation. Others serve supporting roles, such as part-of-speech tagging and syntactic parsing. Solutions draw from machine learning (especially deep learning), algorithms, and linguistics. There is particular interest in structured prediction in which the output structure is a sequence, tree, or sentence.

主题包括:

  • words: lexical semantics, distributional representations, clusters, and word embeddings
  • sequences: language modeling and smoothing, tasks such as part-of-speech tagging and named-entity recognition, model families like hidden Markov models and conditional random fields, neural sequence encoders such as recurrent and convolutional neural networks
  • trees: syntactic parsing, including constituency parsing and dependency parsing, context-free grammars, parsing algorithms
  • computational semantics, including compositionality, reference, and shallow semantic parsing
  • word alignment and machine translation

Assignments include formal exercises as well as practical exercises involving implementing algorithms and using NLP toolkits.

Expected outcomes:

  • Understand key challenges of computing with natural language
  • Understand and apply solutions to standard NLP tasks, such as hidden Markov models, conditional random fields, and bidirectional LSTM taggers for sequence labeling
  • Be able to implement basic neural network architectures for core NLP tasks using deep learning toolkits
  • Be able to derive dynamic programming algorithms to perform inference in structured output spaces, and to analyze their computational properties
  • Understand common types of syntactic and semantic analysis, and how they are used in downstream applications
  • Recognize and characterize the errors made by NLP systems

天天射综合网 Prerequisites: basic knowledge of calculus, linear algebra, and probability; basic programming experience; a machine learning course is recommended but not required.

TTIC 31200 - Information and Coding Theory (CMSC 37220)

100 units

Tulsiani, Madhur

天天射综合网 This course is meant to serve as an introduction to some basic concepts in information theory and error-correcting codes, and some of their applications in computer science and statistics. We plan to cover the following topics:

  • Introduction to entropy and source coding. Some applications of entropy to counting problems.
  • Mutual information and KL-divergence. Method of types and hypothesis testing. I-projections and applications.
  • 纠错码简介。 Unique and list decoding of Reed-Solomon and Reed-Muller codes.
  • Applications of information theory to lower bounds in computational complexity and communication complexity.

Expected outcomes:

  • Familiarity with concepts such as Entropy, Mutual information and KL-divergence.
  • Familiarity with source and channel coding.
  • Understanding of the method of types and ability to derive large-deviation bounds using information-theoretic concepts.
  • Understanding of the notions of unique and list decoding for various codes.

Prerequisites: Discrete probability. Some knowledge of finite-field algebra is required for the part on error-correcting codes but required basics are reviewed in class.

TTIC 31250 - Introduction to the Theory of Machine Learning

100 units

天天射综合网 Blum, Avrim

天天射综合网 This course will cover some of the basic theory underlying machine learning and the process of generalizing from data. We will talk about both the PAC model for batch learning (learning over one set of data with the intent of producing a predictor that performs well on new data) and models for learning from feedback over time (online learning). We will discuss important fundamental concepts including overfitting, uniform convergence, formal notions of Occam's razor, VC-dimension, and regularization, as well as several classic algorithms including the Perceptron algorithm, SVMs, algorithms for combining expert advice, and boosting. We will also discuss limited-feedback (bandit) algorithms, reinforcement learning, connections between learning and game theory, and formal guarantees on privacy. This will be a proof-oriented course: our focus will be on proving performance guarantees for algorithms that aim to generalize from data as well as understanding what kinds of performance guarantees we can hope to prove.

Pre-Requisites:

天天射综合网 The main pre-requisite is comfort with a proof-oriented course, having taken some algorithms class, and comfort with basic probabilistic modeling and reasoning. For example, 1000 programs are submitted to a stock-market prediction challenge, and we find that one of those programs has correctly predicted whether the market will go up or down the next week for 10 weeks in a row; should we feel confident that the program is a good one? Comfort with thinking about points and vectors in high-dimensional space is a plus.

Specific Topics Include:

  • The PAC (Probably Approximately Correct) batch learning model
  • Overfitting and generalization
  • Cardinality and description-length bounds: Occam's razor
  • Regularization
  • The online mistake bound model
  • The Perceptron algorithm
  • Hinge-loss and inseparable data
  • Kernel functions, SVMs
  • Online to batch conversion
  • VC-dimension, the growth function, and Sauer's lemma
  • Uniform convergence proofs using ghost samples
  • Boosting
  • Learning and Game Theory
  • Bandit algorithms
  • 强化学习
  • Differential Privacy
  • Additional possible topics: Semi-Supervised Learning, Rademacher Bounds, Limitations on Learning

Expected outcomes:

  • Ability to recognize different learning models and make rigorous statements about learning methods
  • Ability to use standard techniques to prove learning guarantees
  • Ability to think critically about new learning paradigms

TTIC 31210 - Advanced Natural Language Processing

100 units

天天射综合网 Gimpel, Kevin

This course is a follow-up to TTIC 31190. It will go into more depth of the fundamentals of natural language processing (NLP) and cover a broader range of applications. Some of the class meetings will be hands-on, guided laboratory-style meetings; a laptop is strongly recommended for these class meetings, but not strictly required.

主题包括:

  • grammatical formalisms (CFG, TSG, TAG, and CCG)
  • exact and approximate parsing algorithms (CKY, shift-reduce, k-best parsing, cube pruning, etc.)
  • logical semantics and semantic parsing
  • semantic formalisms (abstract meaning representation, etc.)
  • training and decoding criteria for NLP (eg, minimum Bayes risk)
  • unsupervised learning in NLP (EM for HMMs and PCFGs, topic models, Bayesian nonparametrics)
  • advanced neural network methods in NLP, including recurrent, recursive, and convolutional networks, encoder-decoder architectures, and attention-based models
  • the application of these techniques to important NLP applications, including: textual entailment, dialogue systems, machine translation, question answering, automatic summarization, and coreference resolution

Assignments include formal exercises as well as practical exercises involving implementing algorithms and using NLP and deep learning toolkits.

Expected outcomes:

  • Be able to derive dynamic programming algorithms for inference with grammatical formalisms and other structured output spaces, and to analyze their computational properties
  • Understand trade-offs of approximate inference algorithms used in NLP and be able to choose algorithms appropriately
  • Be able to design generative models for textual data and derive statistical inference algorithms for quantities of interest
  • Understand state-of-the-art solutions to key NLP applications, including approaches based on deep learning

Prerequisites: TTIC 31190 or permission of the instructor.

TTIC 31220 - Unsupervised Learning and Data Analysis

天天射综合网 100 units

Livescu, Karen

天天射综合网 This course will introduce concepts and techniques for analyzing and learning from unlabeled data. Unsupervised methods are used, for example, for visualization, data generation, and representation learning. The course will motivate settings in which unsupervised methods are needed or useful, and discuss relationships between unsupervised and supervised methods. Topics will include fixed feature representations such as Fourier methods and count-based features; linear and nonlinear dimensionality reduction; clustering; density estimation, mixture models, and expectation maximization; and semi-supervised/ distant-supervision settings. Linear, kernel, and deep neural network-based methods will be included. Assignments will include theoretical and practical exercises.

Prerequisites: a good background in basic probability, linear algebra, TTIC 31020, and familiarity and comfort with programming; 或老师的许可。

天天射综合网 Expected outcomes:

  • Understand typical settings where unsupervised methods are used, including visualization, representation, analysis, and generation, and be able to choose relevant methods for a given situation
  • Understand how supervised and unsupervised data and methods can be combined.
  • Be able to analyze and visualize data using relevant fixed feature representations.
  • Understand the motivation and application of dimensionality reduction techniques.
  • Understand and be able to apply clustering and density estimation techniques.
  • Understand the current state of the art and research landscape in selected areas.
  • Develop proficiency in applying relevant techniques to real data in practical settings.

TTIC 31230 - Fundamentals of Deep Learning (CMSC 31230)

100 units

McAllester, David

天天射综合网 Introduction to fundamental principles of deep learning. Although deep learning systems are evolving rapidly, this course attempts to teach material that will remain relevant and useful as the field changes. The course will emphasize theoretical and intuitive understanding to the extent possible.

Lecture Plan:

话题:

  • Introduction to multi-layer perceptrons and backpropagation.
  • Convolutional neural networks (CNNs) and Recurrent neural networks (RNNs).
  • General frameworks and mathematical notations for expressing neural networks.
  • Implementing a framework in Python.
  • The theory and practice of stochastic gradient descent.
  • Batch Normalization
  • Vanishing Gradients and exploding gradients and methods to counter these problems.
    • LSTMs, GRUs, highway networks, and Resnet.
  • Regularization
    • L2 regularization
    • Dropouts
    • Early Stopping
    • Generalization Bounds
  • Model Compression
  • Attention and Memory
  • Stack Architectures
  • 生成对抗网络
  • Autoencoders

Expected outcomes:

  • Ability to design and train novel deep learning architectures.
  • An understanding of the general issues and phenomenon sufficient to guide architecture design and training.

Prerequisites: Introduction to machine learning.

TTIC 31240 - Self-driving Vehicles: Models and Algorithms for Autonomy

天天射综合网 100 units

Walter, Matthew

天天射综合网 This course considers problems in perception, navigation, and control, and their systems-level integration in the context of self-driving vehicles through an open-source curriculum for autonomy education that emphasizes hands-on experience. Integral to the course, students will collaborate to implement concepts covered in lecture on a low-cost autonomous vehicle with the goal of navigating a model town complete with roads, signage, traffic lights, obstacles, and citizens. The wheeled platform is equipped with a monocular camera and a performs all processing onboard with a Raspberry Pi 3, and must: follow lanes while avoiding obstacles, pedestrians and other robots; localize within a global map; navigate a city; and coordinate with other robots to avoid collisions. The platform and environment are carefully designed to allow a sliding scale of difficulty in perception, inference, and control tasks, making it usable in a wide range of applications, from undergraduate-level education to research-level problems. For example, one solitary robot can successfully wander the environment using only line detection and reactive control, while successful point-to-point navigation requires recognizing street signs. In turn, sign detections can be “simulated” either by using fiducials affixed to each sign, or it can be implemented using “real” object detection. Realizing more complex behaviors, such as vision-based decentralized multi-robot coordination, poses research-level challenges, especially considering resource constraints. In this manner, the course is well suited to facilitate undergraduate and graduate-level education in autonomy.

天天射综合网 The course will be taught in concurrently and in conjunction with classes at the University of Montreal and ETH Zurich, which provides opportunities for interaction and collaboration across institutions.

Lecture Plan:

话题:

The course covers fundamental techniques in perception, planning, and control for autonomous agents and their integration as part of a complex system. Specific topics include:

  • camera geometry, intrinsic/extrinsic calibration;
  • minimal sufficient representations for visual tasks;
  • nonlinear filtering, including robust localization and mapping (localize in a given map, or create your own map);
  • shared control and level 2,3 autonomy;
  • complex perception pipelines: (use of) object detection (reading traffic signs) and tracking;
  • safety and correctness (navigate intersections); 和
  • signaling and coordination for distributed robotics (reason about the intent of the other drivers).

Expected outcomes:

  • An understanding of fundamental techniques in perception, planning, and control for autonomous agents and their integration as part of a complex system
  • Understand how to design these subsystems together, such that performance is maximized, while (shared) resource usage is minimized.
  • Familiarity with basic practices of reliable system development, including test-driven and data-driven development.
  • Understanding of the tools and the dynamics of software and hardware open-source development. All homework is shared on a central Git repository.
  • Familiarity with the constraints related to co-design of integrated systems.

天天射综合网 Prerequisites: There are no formal course requirements, though having taken TTIC 31170 – Planning, Learning, and Estimation for Robotics and Artificial Intelligence is desireable. Students must be familiar with the GNU/Linux development environment and are required to have access to a laptop with Ubuntu 16.04 installed.

课程网站

TTIC 41000- Special Topics: Spectral Techniques for Machine Learning

50 units

天天射综合网 Stratos, Karl

A spectral technique refers to a technique that makes use of the eigenvalues of a matrix. Thanks to the rich theory underlying linear algebra and matrix perturbation, these techniques can offer a precise and satisfying understanding of a wide class of problems such as dimensionality reduction and model estimation. Moreover, the existence of powerful algorithms for computing eigenvalues makes the approach also practical, with applications ranging from robotics to natural language processing. This seminar-like course will first supply an inventory of mathematical tools to understand and derive spectral techniques used in modern machine learning. It will apply these tools to examine some of the most recent developments in the literature. In the latter part of the course, the course will “flip” and students will take turns to present a paper on recent research in this area. A student can choose any relevant work under this topic, but personal guidance and references will be provided.

Audience and prerequisites: The course is designed to encourage students to start doing research in this area. Thus the expected audience is graduate students with relevant background, for example a PhD student in machine learning theory/application who is comfortable with proofs and has some basic knowledge of linear algebra (but not familiar with the techniques considered in this course). Both TTIC and UofC students are qualified to take this course, possibly the former getting priority if necessary. Class size is 10-20.

The goals of taking this course are:

  1. Achieving an understanding of the foundational concepts and tools used in modern spectral methods
  2. Obtaining an ability to accurately evaluate new works in this area at conferences
  3. Finding new research projects that persist beyond this course and result in publications

天天射综合网 This course will be pass-fail. All students who actively engage in the class will receive a pass (this will be evaluated based on attendance, asking questions, presentation, etc.).

There will be optional homeworks to reinforce understanding of the materials taught in the class. They will not be graded, but solutions will be posted after a period of time for self-grading.

Course site: http://karlstratos.com/teaching/spectral_topics/spectral_topics.html .

TTIC 55000 - Independent Research

100 units

天天射综合网 Original academic research conducted under guidance of an instructor (normally student's PhD advisor), directed at making progress towards advancing the state of the art in the chosen area.

Expected outcomes:

  • Familiarity with peer-reviewed literature on the chosen topic of focus representing both current state of the art and historical developments.
  • Ability to develop a short-, medium- and long-term research plan.
  • Improved technical skills relevant to the research topic (examples include: using relevant mathematical tools, developing, documenting and using advanced software, designing and executing user studies, designing and conducting experiments).
  • Ability to communicate research progress in regular meetings with advisor and colleagues.
  • Ability to describe research results in a technical document at a level and in format appropriate for a submission to a peer-reviewed venue.

TTIC 56000 - Independent Reading

100 units

Original academic research conducted under guidance of an instructor (normally student's PhD advisor), directed at making progress towards advancing the state of the art in the chosen area.

天天射综合网 Expected outcomes:

  • Familiarity with peer-reviewed literature on the chosen topic of focus representing both current state of the art and historical developments.
  • Ability to develop a short-, medium- and long-term research plan.
  • Improved technical skills relevant to the research topic (examples include: using relevant mathematical tools, developing, documenting and using advanced software, designing and executing user studies, designing and conducting experiments).
  • Ability to communicate research progress in regular meetings with advisor and colleagues.
  • Ability to describe research results in a technical document at a level and in format appropriate for a submission to a peer-reviewed venue.

TTIC 57000 - Computer Science Internship

100 units

顾问

In-depth involvement in areas of computer science in a research lab, University or business setting. Internship activities and objectives must be related to the student's program objectives. Required enrollment for F-1 CPT internship. Advisor's Consent Required.