Thoughts about the Free Lunch

This post starts when I was reviewing the no free lunch theorem in machine learning. A machine learning model being able to predict with high accuracy from unseen data should have knowledge about the data. Then what is knowledge? I started to have the feeling that knowledge should be defined in a broader sense than what it is done in (intro) epistemology textbooks. Namely, it should be closely related to the creation of intelligent systems, which might be more important than defining what knowledge is in human languages.

  • A brief overview about the no free lunch theorem
  • Does learning something indicate having knowledge over it?
  • Gödel’s idea…?
  • What should we do then? (If knowledge is nothing, then what does the hard tasks at schools make sense?)

Read More

Effective Java

This blog contains notes on Joshua Bloch’s Effective Java. I may also include some Java 8 specific entries after that. There are following parts:

  • Creating and Destroying Objects
  • Methods Common to All Objects
  • Classes and Interfaces
  • Generics
  • Enums and Annotations
  • Methods
  • General Programming
  • Exceptions
  • Concurrency
  • Serialization

    Read More

Intro to Philosophy

This blog is based on the notes on Coursera’s Introduction to Philosophy, taught by University of Edinburgh. There are 8 sections:
(1) What is Philosophy? What does “do philosophy” mean?
(2) Morality
(3) Epistemology
(4) Political philosophy
(5) Minds and Machines
(6) Philosophy of Science
(7) Determinism: do we have Free Will?
(8) Time travel.

Read More

Systems Software

This blog is a short review for ECE353 (Systems Software, 2017 winter by Professor Baochun Li) at University of Toronto. This course talks about 7 categories of basic information about computer systems:

  1. Concurrency
  2. Scheduling Policy
  3. Virtualization
  4. File System
  5. I/O
  6. Security
  7. BLITZ

    Read More


This blog is a review of probability information.

  • Probability Distributions
  • Conditional Probability and Bayes Theorem
  • Estimation: ML, MAP
  • From condition probability to Bayesian Networks
  • Example: Naive Bayes Model
  • Example: Bayesian Tracking, Temporal Model and the Plate Model
  • From Directed Model to Undirected Model
  • Exact Inference
  • Message Passing and Belief Propagation

Read More