Skip to main content

Data Science: Data to Insights

Solve Complex Issues With Your Data


Start Date: May 30, 2017
Duration: 6 Weeks
Price: $625

Course Description

Amazing, but true: 90% the world’s data was created in just the past few years. Faced with overwhelming amounts of data, organizations are struggling to extract the powerful insights they need to make smarter business decisions.

To address this challenge, MIT Professional Education has partnered with the MIT Institute for Data, Systems, and Society (IDSS) to offer Data Science: Data to Insights. Specifically designed for data scientists, business analysts, engineers, and technical managers, this in-demand course examines the latest data science techniques through in-depth case studies from Netflix, Amazon, and other data science leaders.

Turn Knowledge into Action :
In addition to learning from MIT’s leading data science experts, you’ll delve into case studies and put your knowledge into action by:

  • Tracking the 2D and 3D position of objects with a Kalman filter
  • Building your own movie, music, and product recommendation systems
  • Automatically clustering news stories with a spectral technique algorithm
  • Predicting wages with a linear regression model
  • Exploring one or two layer perceptrons to assess their decision boundaries
  • Using network-theoretic ideas to identify new candidate genes that might cause autism

What You'll Learn

  • Apply data science techniques to your organization’s data management challenges
  • Identify and avoid common pitfalls in big data analytics
  • Deploy machine learning algorithms to mine your data
  • Interpret analytical models to make better business decisions

Want to purchase this course for a group?

You can purchase enrollment codes for this course to distribute to your team

Email Us

Instructors

Devavrat Shah, Co-Director

Devavrat Shah, Co-Director Director, Statistics and Data Science Center (IDSS), Professor, Laboratory for Information and Decision Systems (LIDS), Computer Science and Artificial Intelligence Laboratory (CSAIL) and Operations Research Center (ORC)

Philippe Rigollet, Co-Director

Philippe Rigollet, Co-Director Associate Professor, Mathematics department and Statistics and Data Science Center (IDSS)

Guy Bresler

Guy Bresler Assistant Professor, Electrical Engineering and Computer Science, LIDS and IDSS

Tamara Broderick

Tamara Broderick Assistant Professor, Institute for Data, Systems, and Society (IDSS), Electrical Engineering and Computer Science (EECS) Department

Victor Chernozhukov

Victor Chernozhukov Professor, Department of Economics; Statistics and Data Science Center (IDSS)

David Gamarnik

David Gamarnik Professor, Sloan School of Management

Stefanie Jegelka

Stefanie Jegelka Assistant Professor, Institute for Data, Systems, and Society (IDSS), Electrical Engineering and Computer Science (EECS) Department

Jonathan Kelner

Jonathan Kelner Associate Professor, Department of Mathematics and a member of the MIT Computer Science and Artificial Intelligence Laboratory (CSAIL)

Ankur Moitra

Ankur Moitra Assistant Professor, Department of Mathematics and member of the Computer Science and Artificial Intelligence Lab (CSAIL)

Caroline Uhler

Caroline Uhler Assistant Professor, Institute for Data, Systems, and Society (IDSS), Electrical Engineering and Computer Science (EECS) Department

COURSE OVERVIEW

By engaging in comprehensive lectures from our MIT IDSS faculty members, you’ll acquire the theories, strategies, and tools you need to convert gigabytes of data into meaningful insights.

Over the course of six weeks, you will review a broad spectrum of topics including recommendation engines, regressions, network and graphical modeling, anomaly detection, hypothesis testing, and machine learning. Using case studies and hand-on exercises, participants will have the opportunity to practice and increase their data analysis skills. After completing this course, you will be well prepared to:

  • Uncover unexpected patterns and anomalies in your data
  • Determine what data you need and how to design experiments
  • Use foundational and emerging analytics techniques
  • Understand common pitfalls in big data analytics and how to avoid them
  • Comprehend how machine learning works in practice
  • Interpret model results and make more effective decisions
  • Overcome the challenges and constraints associated with scaling big data algorithms

You will also receive:

  • 90-day access to archived course materials: Videos, discussion boards, and content
  • Complete course transcript: Synchronized video transcripts and a compiled transcript of all course lectures

Time Commitment
MIT Professional Education Digital Programs are designed to fit the schedules of busy professionals. That’s why this course is self-paced and available online 24 hours a day, 7 days a week.

Each video module is pre-recorded, enabling you to watch it anytime. While you may complete the program as quickly as you wish, most participants find it beneficial to adhere to the weekly schedule and participate in online discussion forums along the way.
The course requires a time commitment of three-to-four hours a week comprised of videos, assigned reading, and assignments.

Browser/Technical Requirements
Access to our courses requires an Internet connection, as videos are only available via online streaming, and cannot be downloaded for offline viewing. Please take note of your company's restrictions for viewing content and/or firewall settings.

About IDSS

MIT Institute for Data, Systems, and Society (IDSS) is committed to addressing complex societal challenges by advancing education and research at the intersection of statistics, data science, information and decision systems, and social sciences. Spanning all five schools at MIT, IDSS embraces approaches and methods from disciplines including statistics, data science, information theory and inference, systems and control theory, optimization, economics, human and social behavior, and network science. These disciplines are relevant both for understanding complex systems, and for presenting design principles and architectures that allow for the systems’ quantification and management. IDSS seeks to integrate these areas—fostering new collaborations, introducing new paradigms and abstractions, and utilizing the power of data to address societal challenges. Read more about IDSS at idss.mit.edu.

EARN A CERTIFICATE OF COMPLETION AND CEUS

CERTIFICATE OF COMPLETION
To earn a Certificate of Completion in this course, participants should watch all the videos, and complete all assessments by the due date, with an overall average of 80 percent success rate. Keep in mind that the 80-percent pass rate is across all assessments, and is your overall average “grade” for the course. 

Upon successful completion of the course and all assessments, a Certificate of Completion will be awarded by MIT Professional Education after the course has ended.

Continuing Education Units (CEUs)

Participants of this course who successfully complete all course requirements in order to earn a Certificate of Completion are eligible to receive 1.3 Continuing Education Units (1.3 CEUs).

CEUs are a nationally recognized means of recording noncredit/non-degree study and are accepted by many employers, licensing agencies, and professional associations as evidence of a participant’s serious commitment to the development of a professional competence.

Acceptance of CEUs depends on the organization to which one is submitting them. If your employer requires any additional information, MIT Professional Education can answer questions and provide information, but we cannot guarantee that any particular organization will accept our CEUs.

CEUs are based on hours of instruction. For example: One CEU = 10 hours of instruction. CEUs may not be applied toward any MIT undergraduate or graduate level course.

WHO SHOULD PARTICIPATE

This course is designed for data scientists and data analysts, as well as professionals who wish to turn large volumes of data into actionable insights. Because of the broad nature of the information, the course is well suited for both early career professionals and senior managers. The course also requires participants to have a substantial background knowledge of statistical techniques and data calculations or quantitative methods of data research.

Participants may include:

  • Technical managers
  • Business intelligence analysts
  • Management consultants
  • IT practitioners
  • Business managers
  • Data science managers
  • Data science enthusiasts

COURSE OUTLINE

Download complete syllabus

The course features five modules:

Module 1: Making sense of unstructured data
Modern businesses, scientific and engineering laboratories, and Web 2.0 generate vast quantities of data, often without existing labels. To make sense of this data, a principal challenge becomes to discover patterns or latent structure where none is known beforehand. For instance, we might want to discover an organic organization of documents, such as articles collected from the New York Times or Wikipedia, into distinct groups representing topics or themes. We might want to discover latent communities in social networks, such as Facebook or Twitter. We might to figure out which aspects of text or images, such as those on Imgur or Google images, capture the important information encapsulated in these data formats. In this module, we offer an overview of modern techniques for addressing these problems across a variety of different types of data. We demonstrate the usefulness of these methods in a number of case studies.

Topics:

  • Clustering
  • Spectral Clustering, Components and Embeddings
  • Case Studies

Module 2: Regression and Prediction
The module provides an introduction to regression, combining both classical and modern views. We will begin with bivariate and multivariate regression for purposes of prediction and causal inference, followed by logistic and nonlinear regression. We then go over a menu of modern prediction methods that aim to solve prediction problems well using high-dimensional data, namely lasso, ridge and various modifications.  We shall discuss regression trees, boosted trees, and random forests, followed by a basic view of neural networks, all for prediction purposes.  We will discuss the assessment of prediction performance using validation samples and cross-validation. We will conclude with a brief discussion of how to use these methods for inferring causal effects of a treatment in randomized control trials and in the presence of confounding.

Topics:

  • Classical Linear & nonlinear regression & extension
  • Modern Regression with High-Dimensional Data
  • The use of modern Regression for causal inference
  • Case Studies

Module 3:  Classification, Hypothesis Testing and Anomaly Detection
This module provides a basic introduction to statistical methods of classification, testing hypothesis and its applications, including detection of statistical anomalies, detection of frauds, spams, and other malicious behaviors. The course will begin by describing informally the range of applications of these techniques and then move on to methods, mostly evolving around the methods of classifications. Those include binary classification, logistic and probit regression, perceptron method and neural networks method, support vector machines, and others. Several examples will be introduced to illustrate the application of the discussed methods. Finally, the course will discuss the limitations of the methods, the importance of careful usage and the dangers of misuse of the discussed methods.

Topics:

  • Hypothesis Testing and Classification
  • Deep Learning
  • Case Studies

Module 4: Recommendation Systems
Recommendation systems have become primary way to discover relevant information from vast amounts of data.  Examples include media recommendations by Netflix, YouTube and Spotify; online dating suggestions by Tinder; news feeds by Facebook; and product recommendations by Amazon and more. This module provides a systematic overview of principles and algorithms for designing and developing recommendation systems. The content is exemplified using concrete case studies.

Topics:

  • Recommendations and ranking
  • Collaborative filtering
  • Personalized recommendations
  • Case Studies
  • Wrap-up: Parting remarks and challenges

Module 5: Networks and Graphical Models
From social networks to gene regulatory networks, networks form the backbone for many of the processes we care about. Local interactions between basic entities in a network give rise to large-scale network effects such as the spread of information or ideas. How do we make use of network data to understand the behavior or functionality of the network? This module provides a systematic overview of methods for analyzing large networks, determining important structure in such networks, and for inferring missing data. An emphasis is placed on graphical models both as a powerful way to model network processes and to facilitate efficient statistical computation. The course content is illustrated via case studies.

Topics:

  • Introduction
  • Networks
  • Graphical Models
  • Case Studies

CASE STUDY OUTLINES

In Data Science: Data to Insights, you won’t just discover new strategies, tools, and insights—you’ll put them to the test. Every course module features a selection of case studies and hands-on projects that help you apply your newfound knowledge to realistic business challenges.

Time Commitment: For participants that wish to engage with the optional case study activities, please allow an extra 4+ hours a week. These Optional Case Study tutorials will require some prior knowledge and experience with the programming language you choose to use for reproducing case study results. Generally, participants with 6 months of experience using “R” or “Python” should be successful in going through these exercises. Please note that the case study activities are not required and do not count towards your "grade" or earning a certificate of completion.

Week 1 - Module 1: Making sense of unstructured data
Faculty Leads: Stefanie Jegelka & Tamara Broderick

Case Study 1: Genetic Codes

  • Case Study Activity Description: Use K-means to figure out that DNA is composed of three-letter words. We’ll help by demonstrating how to apply data visualization to genomic sequence analysis.
  • Data Sets & format: DNA text string
  • Tools used: Matlab

Case Study 2: LDA Analysis

  • Case Study Activity Description: Find themes in project descriptions using LDA. We’ll help by giving you tips on how to do your own analysis on MIT EECS faculty data using stochastic variational inference on LDA.
  • Data Sets & format: Scrape your own
  • Tools used: Python

Case Study 3: PCA: Identifying Faces

  • Case Study Activity Description: Implement your own image classification algorithm that helps classify photos of people’s faces. We’ll help by giving you tips on how to use PCA, along with examples and pseudo-code for the programming environment.
  • Data Sets & format: Instructors photos provided (14). Any other images will work, as long as they obey the restrictions noted in the Self Help document.
  • Tools used: Mathlab

Case Study 4: Spectral Clustering: Grouping News Stories

  • Case Study Activity Description: : Build your own clustering for online news stories—similar to how Google News organizes stories via auto-generated topics. We’ll help by giving you tips on Spectral Clustering, along with examples and pseudo-code for the programming environment.
  • Data Sets & format: Instructions for downloading news stories off the web.
  • Tools used: Python

Week 2 - Module 2: Regression and Prediction
Faculty Leads: Victor Chernuzkov

Case Study 1: Predicting Wages 1

  • Case Study Activity Description: Predict wages and assess predictive performance using various characteristics of workers. We’ll help by describing the wage prediction model.
  • Data Sets & format: CPS 2012 Data, Rdata format
  • Tools used: R

Case Study 2: Gender Wage Gap

  • Case Study Activity Description: Estimate the difference in predicted wages between men and women with the same job characteristics. We’ll help by describing the estimation technique and presenting the results.
  • Data Sets & format: CPS 2012 Data, Rdata format
  • Tools used: R

Case Study 3: Do Poor Countries Grow Faster than Rich Countries?

  • Case Study Activity Description: Use a large dimensional dataset to answer the question: Do poor countries grow faster than rich countries? We’ll help by describing the estimation technique, giving you the tools, and presenting the results.
  • Data Sets & format: Barro-Lee Growth Data. Rdata format.
  • Tools used: R

Case Study 4: Predicting Wages 2

  • Case Study Activity Description: Predict wages using several machine learning methods and splitting data. We’ll help by describing the estimation technique and presenting the results.
  • Data Sets & format: 2015 CPS data, Rdata format.
  • Tools used: R

Case Study 5: The Effect of Gun Ownership on Homicide Rates

  • Case Study Activity Description: Use machine learning methods to estimate the effect of gun ownership on the homicide rate. We’ll help by describing the estimation technique and presenting the results.
  • Data Sets & format: U.S. Census Bureau Dataset. Csv format.
  • Tools used: R

Week 3 - MODULE 3.1: Classification and Hypothesis Testing
Faculty Leads:David Gamarnik & Jonathan Kelner

Case-study 1: Logistic Regression: The Challenger Disaster

  • Case Study Activity Description: Learn how to apply Logistic Regression in a practical real-world setting. We’ll help by giving you tips, examples, and pseudo-code for the programming environments.
  • Data Sets & format: Made available as a csv file along with the case study.
  • Tools used: User Choice: Python or R. Using the statsmodels library or the built-in glm function in R.

Week 4 - MODULE 3.2: Deep Learning
Faculty Leads: Ankur Moitra

Case Study 2: Decision boundary of a deep neural network

  • Case Study Activity Description: Play with one or two layer perceptrons to assess their decision boundaries. We’ll help by explaining the multiple dimensions of perceptrons.
  • Data Sets & format: Synthetic 2D data points.
  • Tools used: Python (coding is not required for students)

Week 5 - MODULE 4: Recommendation Systems
Faculty Leads: Devavrat Shah & Philippe Rigollet

Case Study 1: Recommending Movies

  • Case Study Activity Description: Build your own recommendation system for movies like the one used by Netflix. We’ll help by giving you tips, examples, and pseudo-code for the programming environments.
  • Data Sets & format: MovieLens dataset - public set
  • Tools used: User Choice: Python or R For Recommenders: RecommenderLab and Graphlab-Create

Case Study 2: Recommend New Songs to Users Based on Their Listening Habits

  • Case Study Activity Description: Build your own recommendation system for songs like the one used by Spotify. We’ll help by giving you tips, examples, and pseudo-code for the programming environments.
  • Data Sets & format: Million Song dataset
  • Tools used: User Choice: Python or R For Recommenders: RecommenderLab and Graphlab-Create

Case Study 3: Make New Product Recommendations

  • Case Study Activity Description: Build your own recommendation system for products on an e-commerce website like the one used by Amazon.com. We’ll help by giving you tips, examples, and pseudo-code for the programming environments.
  • Data Sets & format: Amazon Reviews data
  • Tools used: User Choice: Python or R For Recommenders: RecommenderLab and Graphlab-Create

Week 6 - MODULE 5: Networks and Graphical Models
Faculty Leads: Caroline Uhler & Guy Bresler

Case study 1: Navigation / GPS
1.1: Kalman Filtering: Tracking the 2D Position of an Object when moving with Constant Velocity

  • Case Study Activity Description: Generate data, build the model for the motion dynamics, and perform the Kalman Filtering algorithm. We’ll help by giving you tips, examples, and pseudo-code for the programming environment.
  • Data Sets & format: Generating your own data. Model explanation and other parameter details provided in a separate write-up.
  • Tools used: Python. Using libraries like numpy, matplotlib

1.2: Kalman Filtering: Tracking the 3D Position of an Object falling due to gravity.

  • Case Study Activity Description: Generate data, build the model for the motion dynamics, perform the Kalman Filtering algorithm. We’ll help by giving you tips, examples, and pseudo-code for the programming environment.
  • Data Sets & format: Generating your own data. Model explanation and other parameter details provided in a separate write-up.
  • Tools used: Python. Using libraries like numpy, matplotlib

Case study 2: Identifying New Genes that cause Autism

  • Case Study Activity Description:Use network-theoretic ideas to identify new candidate genes that might cause autism. We’ll help by giving you tips, examples, and pseudo-code for the programming environment.
  • Data Sets & format: Made available as csv files.
  • Tools used: R

FAQS

REGISTRATION QUESTIONS

Who can register for this course?
Unfortunately, US sanctions do not permit us to offer this course to learners in or ordinarily residing in Iran, Cuba, Sudan, and the Crimean region of Ukraine. MIT Professional Education truly regrets that US sanctions prevent us from offering all of our courses to everyone, no matter where they live.

What do I need to do to register for the course?
Go to mitprofessionalx.mit.edu, click on the course you would like to register for, and click “Add to Cart.” You may be prompted to first register for a mitprofessionalx account if you do not have one already. Complete this process, then continue with checkout and pay for the course. Once you are given access to the course, the first assignment will be to complete the mandatory entrance survey before you can gain access to the videos and other course materials.

How do I register a group of participants?

    For a group of 5 or more individuals, you can pay via invoice. To be invoiced, please email mitprofessionalx@mit.edu with the number of individuals in your group, and instructions to register will be provided. Please note that our payment terms are net zero, and all invoices must be paid prior to the course start date. Failure to remit payment before the course begins will result in removal from the course. No extensions or exceptions will be granted.

What is the registration deadline?
Individual registrations must be completed by May 30, 2017. For group sales, purchases can take place up until May 23, 2017. Please note that once registration has closed, no late registrations or cancellations will be granted.

How should I pay?
Individual registrants must complete registrations and pay online with a valid credit card at the time of registration. MIT Professional Education accepts globally recognized major credit or debit cards that have a Visa, MasterCard, Discover, American Express or Diner's Club logo. Invoices will not be generated for individuals, or for groups of less than 5 people. However, all participants will receive a payment receipt. Payment must be received in full; payment plans are not available.

When will I get access to the course site?
Instructions for accessing the course site will be sent to all paid registrants via email prior to the course launch date. In order to receive these instructions, please add mitprofessionalx@mit.edu to your “trusted senders” list. If you have not received these instructions by the course start date, visit your account dashboard to login and start the course on the advertised course start date.

Participants are required to provide some personal information via a short mandatory course entrance survey. You will be able to access the survey on the course start date, May 30, 2017. Please be advised that a failure to provide said information will mean that participants will be unable to access course material.
Please see our Terms of Service page for our detailed policies, including terms and conditions of use.

I need to cancel my registration. Are there any fees?
Cancellation requests must be submitted to mitprofessionalx@mit.edu. Cancellation requests received after June 6, 2017 will not be eligible for a refund. To submit your request, please include your full name and order number in your email request. Refunds will be credited to the credit card used when you registered and may take up to two billing cycles to process. MIT Professional Education Digital Programs and edX have no obligation to issue a refund after June 6, 2017.

Can I transfer/defer my registration for another session or course?
Admission and fees paid cannot be deferred to a subsequent session; however, you may cancel your registration and reapply at a later date.

Can someone else attend in my place?
We cannot accommodate any substitution requests at this time. Please review the time commitment section and course schedule

COURSE QUESTIONS

How do I know if this course is right for me?
Carefully review the course description page, which includes a description of course content, objectives, and target audience, and any required prerequisites.

Are there prerequisites or advance reading materials?
The course is open to any interested participant. No advance reading is required. Ability to write code/programming experience not a requirement.

Who will be participating in this course?
Professionals with diverse personal, business, and academic backgrounds from the U.S. and around the world will participate. They include scientists, engineers, technicians, managers, consultants, and others, and they come from industry, government, military, non-profit, and academia.

How long is the course?
The course is held over six weeks, and is entirely asynchronous. Lectures are pre-taped and you can follow along when you find it convenient, as long as you finish all required assignments by July 11, 2017. You may complete all assignments before the due date, however, you may find it more beneficial to adhere to a weekly schedule so you can stay up-to-date with the discussion forums.

What is the time commitment of this course?
MIT Professional Education Digital Programs are designed to fit the schedules of busy professionals. That’s why each course is self-paced and available online 24 hours a day, 7 days a week. Each video module is pre-taped, enabling you to watch it at any time. While you may complete all the assignments in rapid succession, most participants find it beneficial to adhere to the weekly schedule and participate in online discussion forums along the way. There are approximately two hours of video every week. You will spend additional time on multiple choice assessments, readings, and discussion forums. Most participants will spend about 3 - 4 hours a week on course-related activities.

For participants that wish to engage with the optional case study activities, please allow an extra 4+ hours a week. These Optional Case Study tutorials will require some prior knowledge and experience with the programming language you choose to use for reproducing case study results. Generally, participants with 6 months of experience using “R” or “Python” should be successful in going through these exercises. Please note that the case study activities are not required and do not count towards your "grade" or earning a certificate of completion.

How long will the course material be available online?
The materials will be available to registered and paid participants for 90 days after the course end date, October 9, 2017. No extensions may be granted.

What reference materials will be available at the end of the course?
Participants will have 90-day access to the archived course (includes videos, discussion boards, content, and Wiki).

What materials will participants keep at the end of the course?
Participants will take away program materials, and resources presented in the course Wiki, including downloadable case study activities for you to work on in your spare time during or after the course.

Will I receive an MIT Professional Education Certificate?
Participants who successfully complete the course and all assessments will receive a Certificate of Completion. This course does not carry MIT credits or grades, however, an 80% pass rate is required in order to receive a Certificate of Completion.

Will I receive MIT credits?
This course does not carry MIT credits. MIT Professional Education offers non-credit/non-degree professional programs for a global audience. Participants may not imply or state in any manner, written or oral, that MIT or MIT Professional Education is granting academic credit for enrollment in this professional course. None of our Digital courses or programs award academic credit or degrees. Letter grades are not awarded for this course.

Will I earn Continuing Education Units (CEUs)?
Course participants who successfully complete all course requirements are eligible to receive 1.3 Continuing Education Units (CEUs) from MIT Professional Education.CEUs are a nationally recognized means of recording non-credit/non-degree study. They are accepted by many employers, licensing agencies, and professional associations as evidence of a participant’s serious commitment to the development of a professional competence. CEUs are based on hours of instruction. For example: One CEU = 10 hours of instruction. CEUs may not be applied toward any MIT undergraduate or graduate level course.

After I complete this course, will I be an MIT alum?
Participants who successfully complete a Digital Programs course are considered MIT Professional Education Alumni. Only those who complete an undergraduate or graduate degree are considered MIT alumni.

Are video captions available?
Each video for this course has been transcribed and the text can be found on the right side of the video when the captions function is turned on. Synchronized transcripts allow students to follow along with the video and navigate to a specific section of the video by clicking the transcript text. Students can use transcripts of media-based learning materials for study and review. In addition, we include a complete course transcript in a single PDF file that allows for easy reference.

Browser/Technical Requirements
Access our courses requires an Internet connection, as videos are only available via online streaming, and cannot be downloaded for offline viewing. Please take note of your company's restrictions for viewing content and/or firewall settings. Our courseware works best with current versions of Google Chrome, Firefox, or Safari, or with Internet Explorer version 10 and above. For the best possible experience, we recommend switching to an up-to-date version of Chrome. If you do not have Chrome installed, you can get it for free here: http://www.google.com/chrome/browser/
We are unable to fully support access with mobile devices at this time. While many components of your courses will function on a mobile device, some may not.

I have never taken a course on the edX platform before. What can I do to prepare?
Prior to the first day of class, participants can take a demonstration course on edx.org that was built specifically to help students become more familiar with taking a course on the edX platform.

What are the technical requirements to participate in this course?
Our courseware works best with current versions of Google Chrome, Firefox, or Safari, or with Internet Explorer version 10 and above. For the best possible experience, we recommend switching to an up-to-date version of Chrome. If you do not have Chrome installed, you can get it for free here: http://www.google.com/chrome/browser/

Enroll