diff --git a/content/_index.md b/content/_index.md index 7ac5da8ec..3688f5635 100644 --- a/content/_index.md +++ b/content/_index.md @@ -12,5 +12,3 @@ title: "Jet Hughes" - [[notes/info-203]] ## 2 Other - -- [daily notes](notes/daily-notes) \ No newline at end of file diff --git a/content/notes/10-heaps-and-heapsort.md b/content/notes/10-heaps-and-heapsort.md new file mode 100644 index 000000000..59d626c9a --- /dev/null +++ b/content/notes/10-heaps-and-heapsort.md @@ -0,0 +1,8 @@ +--- +title: "10-heaps-and-heapsort" +tags: +--- + +# 10-heaps-and-heapsort + + diff --git a/content/notes/analysis-of-recursive-algorithms.md b/content/notes/analysis-of-recursive-algorithms.md new file mode 100644 index 000000000..8e0b57d19 --- /dev/null +++ b/content/notes/analysis-of-recursive-algorithms.md @@ -0,0 +1,125 @@ +--- +title: "analysis-of-recursive-algorithms" +tags: cosc201 +--- + +# analysis-of-recursive-algorithms + +- induction and recursion are linked +- inductive approach is esential for understanding time-complexity of resursive algorithms + +## 1 Proof by induction +[[Induction]] +Find a (positive integer) _parameter_ that gets smaller in all recursive calls +Prove inductively that "for all values of the parameter, the result computed is correct" +To do that: +- check correctness is all non-recursive cases +- check correctness in recursive cases assuming correcness in the recursive calls + +## 2 Examples +### 2.1 Quicksort +[[divide and conquer]] algorithm +sorts a range in an array (a group of elements between some lower index, $lo$ inclusive and some upper index $hi$ exclusive) as follows: +- If length of range $(hi - lo)$ is at most 1 -> do nothing +- otherwise, choose a pivot p (e.g., the element at $lo$) and: + - place all items less that p in positions $lo$ to $lo +r$ + - place all items >= p in positions $lo +r+1$ to $hi$ + - place p in position $lo+r$ + - call quicksort on the ranges $lo$ to $lo + r$ and $lo+r+1$ to $hi$ + +#### 2.1.1 Proof +parameter is $hi - lo$ + +the parameter gets smaller in all recusive call because we always remove the element $p$ so, even if it is the smallest or largest element of the range ,,the recursive call has a range of size at most $hi - lo - 1$ + +the non-recursive case is correct because if we have 1 or fewer elements in a range they are already sorted + +in the recirsive case, since all the elements before $p$ are smaller than it and we assume they get sorted correctly be quicksort, and the same happens for the elements larger than p, we will get a correctly sorted array + + +### 2.2 Fibonacci 1 +```python + def fib(n) + if n <= 1 + return 1 + return fib(n-1) + fib(n-2) +``` + +line 1 -> always executed +line 2 -> executed if n<=1 +line 4 -> executed if n>1, cost equal to cost of callling fib(n-1), fib(n-2), and some constant cost for the addition and return + +#### 2.2.1 Cost bounds/Proof +if we let T(n) denote the time required for evaluating fib(n) using this algorithm this analysis gives: + +>## $T(0) = T(1) = C$ +>## $T(n) = D + T(n-1) + T(n-2)$ + +where c and d are some positive (non-zero) constants. + +- this shows that T(n) grows at least as quick as fib(n) +- even if $D=0$ we'd get $T(n) = C \times fib(n)$ +- growth rates are the same $\therefore$ exponential (at least $1.6^n$) and far too slow + +> A recurive algorithm that makes two or more recurive calls with parameter values close to the original will generally have exponential time complexity + +### 2.3 Fibonacci 2 +```python + def fibPair() + if n == 1 + return 1, 1 + a,b = fibpair(n-1) + return b, a+b +``` +line 1 -> always executed some constant cost +line 2-> executed if n=1, some constant cost +line 4-> executed if n>1, cost equal to cost of calling fibPair(n-1) +line 5 -> executed if n>1, some constant cost + +#### 2.3.1 Proof +it's true for $n-1 by design$ +If it's true at n-1 then the result of computing fibpair(n) is: + +$(f_{n-1}, f_{n-1} + f_{n-1}) = (f_{n-1}, f_n)$ + +which is what we want + +#### 2.3.2 Cost bounds +if we let P(n) denote the time required for evaluating fib(n) using this algorithm this analysis gives: + +$P(1) = C$ +$P(n) = P(n-1) + D\ for\ n>1$ + +where $C$ and $D$ are some positive (non-zero) constants. + + + Claim: $P(n) = C + D(n-1)$ + +By induction: +it's true for n = 1 since, + +$P(1) = C$ +$C+D\times(1-1)=C$ + +suppose that it's true for n-1. Then it's true for n as well because + +$P(n) = P(n-1) + D$ +$\ \ \ \ \ \ \ \ \ = C+D\times(n-2)+D$ +$\ \ \ \ \ \ \ \ \ = C+D\times(n-1)$ + +$\therefore$ By induction it's true for all $n>=1$ + + + +$P(n)$ is the time for evaluating $fibPair(n)$ using this algorithm. This analysis gives: + +$P(1) = C$ +$P(n) = P(n-1) +D$ + +where C and D are some positive constants + +#theorem +> ## $P(n) = C+D\times(n-1)$ +> in particular, $P(n) = \theta(n)$ + +> A recursive algorithm that make one recurive call with a smaller value and a constant amount of additional work will have at most linear time complexity diff --git a/content/notes/big-o.md b/content/notes/big-o.md new file mode 100644 index 000000000..61f75cbab --- /dev/null +++ b/content/notes/big-o.md @@ -0,0 +1,17 @@ +--- +title: "big-o" +tags: cosc201 +--- + +# big-o + +>Big O means $f(n) = O(g(n))$ if there is some constant $A > 0$ such that for all sufficiently large n, $f(n) ≤ A × g(n).$ + +- Big O provides *upper bounds* only. (usually on worst case runtimes) + - sometimes cost will be much less + - does not take special cases into account + - upper bound +- $O$ says that $g(n)$ provides an upper bound for $f(n)$ + - "Insertion sort is $O(n^2)$" -> the maximum number of basic operations in never more than some constanct times $n^2$ +- if $f(n) =O(g(n))$ then the opposite is also true +- usually $f(n)$ is complex but $g(n)$ is very simple \ No newline at end of file diff --git a/content/notes/big-theta.md b/content/notes/big-theta.md new file mode 100644 index 000000000..c9e924c6d --- /dev/null +++ b/content/notes/big-theta.md @@ -0,0 +1,16 @@ +--- +title: "big-theta" +tags: cosc201 +--- + +# big-theta + +>Big theta means $f(n) = \Theta(g(n))$ if there are constants 0 < B < A such that for all sufficiently large n, ==$B × g(n) ≤ f(n) ≤ A × g(n)$== + +- Upper and lower bound +- $Θ$ says that $g(n)$ provides **upper** and **lower** bound for $f(n)$ + - "selection sort is $\Theta(n^2)$" -> the maximum number of operations will be bounded bothh above and below by some constant times $n^2$ +- $f(n) = \Theta(g(n))$ means that f and g have similar growth rates +- if $f(n) = \Theta(g(n))$ then the opposite is also true +- usually $f(n)$ is complex but $g(n)$ is very simple + diff --git a/content/notes/cosc-201-outline.md b/content/notes/cosc-201-outline.md new file mode 100644 index 000000000..0ab7d3a14 --- /dev/null +++ b/content/notes/cosc-201-outline.md @@ -0,0 +1,18 @@ +--- +title: "cosc-201-outline" +tags: cosc201 outline +--- + +# cosc-201-outline + +- [[big-o]] +- [[big-theta]] +- [[induction]] +- [[analysis-of-recursive-algorithms]] +- [[union-find]] +- [[heap]] +- [[sorting]] +- [[heapsort]] +- [[mergesort]] +- [[quicksort]] + diff --git a/content/notes/cosc-202-lectures.md b/content/notes/cosc-202-lectures.md index e85716f85..0ed5a7251 100644 --- a/content/notes/cosc-202-lectures.md +++ b/content/notes/cosc-202-lectures.md @@ -1,5 +1,6 @@ --- title: "cosc-202-lectures" +tags: lectures cosc202 --- # Cosc 202 Lectures diff --git a/content/notes/cosc-202-outline.md b/content/notes/cosc-202-outline.md new file mode 100644 index 000000000..7521247b8 --- /dev/null +++ b/content/notes/cosc-202-outline.md @@ -0,0 +1,8 @@ +--- +title: "cosc-202-outline" +tags: cosc202 outline +--- + +# cosc-202-outline + + diff --git a/content/notes/heap.md b/content/notes/heap.md new file mode 100644 index 000000000..60c874236 --- /dev/null +++ b/content/notes/heap.md @@ -0,0 +1,32 @@ +--- +title: "heap" +tags: cosc201 datastructure +--- + +# heap + +A tree where: + +1. every elements should be greater than ites children +2. the structure should be filled from top to bottom and left to right + + +To remove an element + +- remove from the top, replace with the last element +- to fix the first condition swap the top element with the highest of its children until fixed + + +To Add an element + +- add to the next position +- If its larger than its parent then swap them + + +How deep is the tree? + +- each layer is twice as deep and the preceding one +- layer k can hold $2^k$ elements +- to store n elements we use k layers where $k = lg n$ +- so we need ϴ(lg n) layers +- So any algorithm that 'walk along a branch' in while or in part will have Ο(n) complexity (assuming constant time work at each node) diff --git a/content/notes/induction.md b/content/notes/induction.md new file mode 100644 index 000000000..2ebb29cb1 --- /dev/null +++ b/content/notes/induction.md @@ -0,0 +1,70 @@ +--- +title: "induction" +tags: cosc201 +--- + +# induction + +# Induction +## 1 PECS +Phases of argument by induction + +- Preparation -> most important +- Execution -> becomes routine if prep is good +- Checking -> second most important +- Satisfaction + +### 1.1 Preparation +- isolate the property that you are trying to verify and the parameter, n, associated with is + - e.g., min possible size of set of rank k is $2^n$ +- Confirm by hand that for small values of the parameter, the property is true +- Use previous cases as assumptions +- Pause and reflect +- If you understand what's going on -> proceed to execution + +### 1.2 Execution +Technical and prescribed (once you're an expert you can take some liberties) + +Four parts +- statement +- verificatio of base case +- inductive step +- conclusion + +e.g., +- we will prove that, for every non-negative integer $n$, *insert property here* +- For $n = 0$, *The property* is true because *explicit verification of this case* +- for any $n > 0$, assuming *the property* is true for $n-1$ (or, for all $k < n$), *the property* is true at $n$ because *explain why we can take a step up* +- Therefore, by induction, *the property* is true for all n. + +### 1.3 Checking +Basically debugging without a compiler to find errors +- have you forgotten anything? e.g., the base case +- Does the inductive step work fro 0 to 1? or are they irregular +- Make sure that you are only assuming the result for things less than $n$ +- ideally show someone and try to convince them (dont let them be polite) +- if necessary go back to execution or preparation + +### 1.4 Satisfaction +Commence satisfaction. +Confidence +100. 😆 + +## 2 Examples +### 2.1 Union Find - min size for set of rank k + +- Initially every element is its own representative and every element has rank 0; +- when we do a union operation, the the two reps have different ranks, the ranks stay the same +- when we do a union operation, if the two reps have the same rank, then the rank increases + +minimum (and only) size of a rank 0 rep is 1 + +to get a rank 1 representative, we form a union of either a rank 0 and a rank 1 set or two rank 0 sets +for the minimum possible size, it must be the second case, and the two rank 0 sets must be each of minimum size 1, so this gives minimum size for a rank 1 set of 2 + +To get a rank 2 rep, we form a union of either rank 2 and rank 0 or 1 set, or two rank 1 sets +For the minimum possible size, it must be the second cae, and the two rank 1 sets must each be of minimum size 2, so this gives minimum size for a rank 2 set of 4 + +To get a rank $n$ rep, we form a union of either rank $n$ and rank $k$ set for some $k $2^{n-1} + 2^{n-1} = 2\times2^{n-1} = 2^n$ diff --git a/content/notes/union-find.md b/content/notes/union-find.md new file mode 100644 index 000000000..3feb5d151 --- /dev/null +++ b/content/notes/union-find.md @@ -0,0 +1,172 @@ +--- +title: "union-find" +tags: cosc201 datastructure +--- + +# union-find + +## 1 Example +- We have 12 'objects' +- *Some* pairs have been connected +- Nodes with a sequence of edges between them form a group + - e.g., 0 5, 2, 1 4 6 9, 3 8 10 11, 7 + - +![](https://i.imgur.com/9iRxZoh.png) + +- Groups with no connecting edges are *disjoint* sets + +## 2 Requirements +- Make(n) - make a set of n vertices with no edges between them +- Union(x, y) - connect x and y by an edge (merge their two groups) + - y becomes the representative node for the whole group + - e,g,. Union(2, 1) + - now : 0 5 2 1 4 6 9 3 8 10 11 7 + - the representative node of the new group is 1 + - the number of groups is always : n - number of union operations between elements of different groups +- Find(x) Find and return a representative of the group the x belongs to. + - If x and y are in the same group then Find(x) == Find(y) + - + +## 3 Implementation +### 3.1 UF 1 + +```java + int[] reps; + public void make(int n){ + reps = new int[n]; + for(int = 0; i < n; i++) reps[i] = i; + } + + public int find(int x){ + return reps[x]; + } + + public void union(int x, int y){ + rx = reps[x] + ry = reps[y] + for i = 0 to n-1 + if reps[i] = rx then + reps[i] = ry + end if + end for + } +`````` + + + Operation | Cost | reason +-------------|------| -- + make | $\Theta(n)$ | filling n place of an array + find(x) | $\Theta(1)$ | find value in array is constant + union(x, y) | $\Theta(n)$ | When x and y's rep are different, the whole array must be examined + +Total possible number of union calls where x and y's rep are different is n-1 +So the Total possible cost of all union calls is $\theta(n^2)$ + +### 3.2 UF 2 + +``` java + int[] reps; + public void make(int n){ + reps = new int[n]; + for(int = 0; i < n; i++) reps[i] = i; + } + + public int find(int x){ + if(reps[x]==x) return x; + return find(reps[x]); + } + + public void union(int x, int y){ + reps[find(x)] = find(y); + } +``` + +Operation | Cost | reason +-------------|------| -- + make | $\Theta(n)$ | filling n place of an array + find(x) | $\Theta(n)$ | need to look through chain nodes for representative + union(x, y) | $\Theta(n)$ | bounded by two calls to find + +Total possible number of union calls where x and y's rep are different is n-1 +So the Total possible cost of all union calls is $\theta(n^2)$ + +### 3.3 UF 3 + +For each rep, let its rank be the length of the longest chain of local reps that reaches it +When union(x,y) make the rep with the larger rank the rep of the other +If equal ranks -> make the second the rep of the first + +``` java + int[] reps; + int[] rank; + public void make(int n){ + reps = new int[n]; + rank = new int[n]; + for(int = 0; i < n; i++) reps[i] = i; + } + + public int find(int x){ + if(reps[x]==x) return x; + return find(reps[x]); + } + + public void union(int x, int y){ + rootUnion(find(x), find(y)) + } + + //x and y are known to be representatives + private void rootUnion(x, y){ + if(rank[x] > rank[y]){ + reps[y] = x; + } else if (rank[y] > rank[x]){ + reps[x] = y; + } else { //rank[x] == rank[y] + reps[x] = y; + rank[y] ++; + } + } +``` + +Operation | Cost | reason +------------|------| -- + make | $\Theta(n)$ | filling n place of an array + find(x) | $\Theta(lg\ n)$ | rank is bounded by $lg\ n$ + union(x, y) | $\Theta(lg\ n)$ | bounded by two calls to find + +Total possible number of union calls where x and y's rep are different is n-1 +So the Total possible cost of all union calls is $\theta(n^2)$ + +trade off means this requires an extra $\theta(n)$ space + +#### 3.3.1 Min size of set of rank k + +- for k = 0 -> size must be at least 1 +- for k = 1 -> size must be at least 2 + +for larger k -> the set must have been formed by the union of two sets of rank k-1. So its size must be at least twice the min size of a set of rank k-1 + +--> min size of set of rank k is $2^k$ + +#theorem +>a set of rank k must contain at least $2^k$ elements + +$\therefore$ The maximum rank of an element is $\log_2(n)$ -> $lg(n)$ + +since the time for $Find$ is big-$\theta$ of the rank of the representative found we get $O(lg n)$ bounds for both find and union + +^we used $O$ not $\theta$ because we dont know that the worst case will always occur. + +If could happen that the sequence of Union operations does not create a rank that is as big as i could be + +^this is an example of a semi-formal proof by [[Induction]] + +### 3.4 UF 4 +Change find so it implements [[path compression]] to "flatten" the chains + +```java + if (x != reps[x]) { + reps[x] = find(reps[x]); + } + return reps[x]; +``` + diff --git a/content/templates/note-header.md b/content/templates/note-header.md index 76793b497..267dea5c4 100644 --- a/content/templates/note-header.md +++ b/content/templates/note-header.md @@ -1,5 +1,6 @@ --- title: "<% tp.file.title %>" +tags: --- # <% tp.file.title %>