Reviewing Calculus: Derivatives - Part 1

Continuing from the review of limits, this post will be the first of a many part review of derivatives. There's so much to cover in derivatives its tough to know what all to include and where to start. I'll try to cover enough that you will be able to take the derivative of just about anything by working/reading through these posts.

I'll begin by stating the definition. A derivative is the slope of the tangent line, of a function; it is the rate of change at a given point in a function. Wikipedia defines it as, a measure of how a function changes as its input changes. The mathematical definition is as follows:$$\lim \limits_{\Delta x \to 0}\frac{f(x + \Delta x) - f(x)}{\Delta x}$$So, let's talk about differentiation.

Given the function $f(x)$, $f'(x)$ is it's derivative. That is to say, $$f'(x) = \lim \limits_{\Delta x \to 0}\frac{f(x + \Delta x) - f(x)}{\Delta x}$$Note: One of the other common derivative notation's - For $y = f(x)$, the derivative can be written as $\frac{dy}{dx}$ or $\frac{d}{dx}(f(x))$

Let's look at a few examples.

Differentiate the functions,  $f(x) = x^2 + 24$ and $g(x) = 4x^3 - 9x + 15$. 

Starting with $f(x)$, we can substitute the function into the limit and evaluate it.$$f'(x) = \lim \limits_{\Delta x \to 0}\frac{((x + \Delta x)^2 + 24) - (x^2 + 24)}{\Delta x}=\lim \limits_{\Delta x \to 0}\frac{x^2 + 2x\Delta x + \Delta x^2 + 24 - x^2 - 24}{\Delta x}$$ $$= \lim \limits_{\Delta x \to 0}\frac{2x\Delta x + \Delta x^2}{\Delta x}= \lim \limits_{\Delta x \to 0}\frac{\Delta x(2x + \Delta x)}{\Delta x}= \lim \limits_{\Delta x \to 0}2x + \Delta x = 2x$$Same plan with $g(x)$. $$g'(x) = \lim \limits_{\Delta x \to 0}\frac{(4(x + \Delta x)^3 - 9(x + \Delta x) + 15) - (4x^3 - 9x + 15)}{\Delta x}$$ $$=\lim \limits_{\Delta x \to 0}\frac{(4(x^3 + 3x^2\Delta x + 3x\Delta x^2 + \Delta x^3) - 9x - 9\Delta x + 15) - 4x^3 + 9x - 15}{\Delta x}$$ $$= \lim \limits_{\Delta x \to 0}\frac{4x^3 + 12x^2\Delta x + 12x\Delta x^2 + 4\Delta x^3 - 9x - 9\Delta x + 15 - 4x^3 + 9x - 15}{\Delta x}$$ $$= \lim \limits_{\Delta x \to 0}\frac{12x^2\Delta x + 12x\Delta x^2 + 4\Delta x^3 - 9\Delta x}{\Delta x}= \lim \limits_{\Delta x \to 0}\frac{\Delta x(12x^2 + 12x\Delta x + 4\Delta x^2 - 9)}{\Delta x}$$ $$= \lim \limits_{\Delta x \to 0}12x^2 + 12x\Delta x + 4\Delta x^2 - 9 = 12x^2 - 9$$We have now successfully differentiated two functions: $f'(x) = 2x$ and $g'(x) = 12x^2 - 9$.

At this point, a pattern may have become apparent. Regardless, shortcuts exist for taking derivatives but, it's important to understand how they work first. Since we've worked a few examples of this style of derivative already, the first one we will be looking at is for functions of the form $f(x) = x^a$. The method for this is as follows, $f'(x) = ax^{a-1}$. This should be clear after a few examples.

Let's differentiate the following: $f(x) = x^7$, $g(x) = 3x^3 + 6$ and $h(x) = -x^2$: $$f'(x) = (7)x^{7-1} = 7x^6$$ $$g'(x) = (3)3x^{3-1} + (0)6^{0-1} = 9x^2$$ $$h'(x) = (2)-x^{2-1} = -2x$$Note: It may be easier to ignore constants in functions when taking derivatives. Since, we are taking the derivative with respect to a variable ($x$ in our examples), parts of the function that don't have that variable can be effectively ignored. For example, when taking the derivative of $f(x) = 3x + 4$, you can ignore the $4$ since it's not affecting the variable.

Hopefully, that was relatively straight-forward. We're going to stop here for now. In the next post, we'll go over adding, subtracting, multiplying and dividing derivatives. Afterward, we'll move on to the trigonometric, logarithmic and exponential derivatives and some application. Eventually, after derivatives have been explained well, we'll move on to some more difficult areas of calculus.

Any questions, comments and/or suggestions are appreciated. 

Reviewing Calculus: Limits

It's been a while since I've formally done any sort of calculus. Before moving on to more advanced topics in math, it's probably worth doing a minor review of the topic.

Starting with the basics, limits, we will work out a simple problem.
Find the tangent line of: $f(x) = -x^2 + 6\,\!$ where $x = 2$.

I guess, it may help to first define what a tangent line is and also a secant line since we will be using them both:
Tangent Line - A straight line is considered a tangent line of a curve if it goes through the point $(x, f(x))$.
Secant Line - A line is considered a secant line of a curve if it intersects two points on that curve. Assuming one of these points is $(x, f(x)$, these two points brought together, will give you the tangent line.

Now, in order to get a [tangent] line, we are going to need a slope. And, in order to get a slope, we're going to need two points. We already know how to get the first point (the point the tangent line must pass through) by solving for $(x, f(x))$ which, in this case, will be $(2, f(2))$ where, $f(2) = -2^2 + 6 = 2$, giving us our first point at $(2,2)$.

From here we can take our next point (for our secant line) at many different locations but, to keep things positive and simple, we'll use $x = 1$. This works out to $(1, f(1))$ where, $f(1) = -1^2 + 6 = 5$, giving us our second point at $(1, 5)$.

Taking the slope of these two points $\frac{5 - 2} {1 - 2}$, we now have a secant line (with a slope of $-3$) to work with. (Just to be clear, the line is $y = -3x + 8$)

As stated in the definitions, we can get the slope of the tangent line by bringing the points on the secant line closer together; more specifically, as the point $(1, 5)$ approaches the point $(2, 2)$, the closer the secant line gets to matching our tangent line. Using this knowledge, we can look at values of $x$ for our slope that are getting closer to $2$, such as $(1.5, f(1.5))$, $(1.8, f(1.8))$, and $(1.95, f(1.95))$. Solving for them we get, $-3.5$, $-3.8$, and $-3.95$ respectively.

Note: We can't use $2$ because we can't divide by zero. Also, it was an option to approach from the other side of the point using a point like $(3, f(3)) \Rightarrow (3, -3)$ and bring it closer to $(2, 2)$; this would give us the same result in this case but, that is not always the case as we will see later.

As you can see, the closer we get to the point $(2, 2)$, the closer we get to a slope of $-4$.
Using this slope, we can now express the tangent line as $y = -4x + 10$.

Now, expanding on this problem, $f(x) =$ some rate of change over period $x$. We can calculate this rate of change at different points using $\frac {f(x) - f(a)} {x - a}$, where $a =$ the point we are trying to reach. As $x$ approaches $a$, this point is called the Instantaneous Rate of Change.
For example, if we want to know the instantaneous rate of change at $1$ $(a = 1)$, we can follow these steps: $$\frac{f(x) - f(1)}{x - 1} \Rightarrow \frac{-x^2 + 6 - 5}{x - 1}$$ And, as $x$ approaches $1$ from both sides, we get closer and closer to $-2$.

Looking at the similarities in these two processes, you can infer that the instantaneous rate of change is also the same as the slope of the tangent line for a given function. Which leads us to the next point...

When you are trying to find the slope of a tangent line for a function, given a secant line, we only need to know one things, what point is $x$ trying to approach ($a$)? Now, we don't really need a secant line (explicitly, anyways) and without that, the arbitrary $a$ is not really needed either. All we really need to know is, what is the change in $f(x)$, represented by: $$(f(x + the distance from  x) - f(x))$$ over the change in $x$, represented by: $$(x + the distance from  x - x)$$The distance from $x$ can be represented by $\Delta x$. This looks like, $$\frac{f(x + \Delta x) - f(x)} {\Delta x}$$as $\Delta x$ approaches $0$.

In fact, we have a mathematical way of saying all of that, which is the limit: $$\lim \limits_{\Delta x \to 0}\frac{f(x + \Delta x) - f(x)}{\Delta x}$$This is also the mathematical definition of a derivative.

Now, the next calculus post will talk about derivatives so, let me just finish off here with a few examples of limits.

There are a few topics in limits I want to cover before moving on to derivatives: 3 methods of computing a limit and limits that you cannot evaluate. One thing to also keep in mind is that the limit of a function is NOT telling you what a function is at a point, rather what the function looks like as it approaches a point. This is important to understand as we move on.

In this first example, we will talk about the 3 methods of computing a limit. The three methods are: the substitution, the factoring and the conjugate method. The substitution method speaks for itself (substitute the variables and solve) so, I will combine it with the factoring method. Let's look at the limit: $$\lim \limits_{x \to 4}\frac{x^2 - 16}{x - 4}$$As you can see, we can't just plug in 4 for x so, we must first do some basic factoring: $$\lim \limits_{x \to 4}\frac{x^2 - 16}{x - 4}\Rightarrow \lim \limits_{x \to 4}\frac{(x + 4)(x - 4)}{x - 4}\Rightarrow \lim \limits_{x \to 4}x + 4$$Now, we can just substitute and see that: $$\lim \limits_{x \to 4}x + 4 = 8\Rightarrow \lim \limits_{x \to 4}\frac{x^2 - 16}{x - 4} = 8$$ When we can't substitute right away or factor, we turn to the conjugate method. Let's look at the limit: $$\lim \limits_{x \to 25}\frac{\sqrt{x} - 5}{x - 25}$$By multiplying by the conjugate we end up with: $$\lim \limits_{x \to 25}\frac{\sqrt{x} - 5}{x - 25} * \frac{\sqrt{x} + 5}{\sqrt{x} + 5}\Rightarrow \lim \limits_{x \to 25}\frac{(\sqrt{x} - 5)(\sqrt{x} + 5)}{(x - 25)(\sqrt{x} + 5)}\Rightarrow \lim \limits_{x \to 25}\frac{x - 25}{(x - 25)(\sqrt{x} + 5)}$$ $$\Rightarrow \lim \limits_{x \to 25}\frac{1}{\sqrt{x} + 5}$$And again, we can just substitute from here and see that: $$\lim \limits_{x \to 25}\frac{1}{\sqrt{x} + 5} = \frac{1}{10}\Rightarrow \lim \limits_{x \to 25}\frac{\sqrt{x} - 5}{x - 25} = \frac{1}{10}$$Finally, limits that do not exist. Let's look at the limit:$$\lim \limits_{x \to 0}g(x)$$where, $$when  x < 0,  g(x) = x  and  when  x \geq 0,  g(x) = x + 1$$Evaluating the limit for this is a little different than usual. In this case we will want to evaluate the limit from each side of $x$. When we want to take the limit from the left side, we denote it with a '-' and, when we want to take the limit from the right side, we use a '+'. For example:$$\lim \limits_{x \to 0^-}g(x)  or  \lim \limits_{x \to 0^+}g(x)$$These limits can be easily evaluated to $0$ from the left and $1$ from the right but, what about the standard limit?$$\lim \limits_{x \to 0}g(x)$$This limit does not exist (DNE) because the function is not continuous (it breaks at 0 in this case). One more example of this to try and make it clear.

Depending on the function and what value we're approaching, what the limit evaluates to can be quite different. Let's look at the limit:$$\lim \limits_{x \to 0}\frac{1}{x}$$In this example, both the left and and right hand limits exist, respectively giving us $-\infty$ and $\infty$. However taking the normal limit (from both sides) does not work. We see that:$$\lim \limits_{x \to 0}\frac{1}{x} = DNE$$This is again because, the function is not continuous. Graphing out these functions should help make this concept clear.

There are more to limits than what I've explained here but, hopefully this was a solid review. Also note that there are more methods available to help solve limits, such as L'Hôpital's rule, that you should check out when you have the chance. On my next calculus post, I will build on these concepts to review derivatives and some related concepts.

Here is a site for an online graphing calculator.

Any questions, comments and/or suggestions are appreciated.

Linked Lists

For this example, the text used was:
Data Structures and Algorithms:
Annotated Reference with Examples
First Edition
Copyright Granville Barnett, and Luca Del Tongo 2008.
Available at: dotnetslackers.com

One of the more basic data structures you come across as you learn to program are linked lists. Linked lists are basically just a collection of nodes that contain a value and a reference to the next node in the list. In order to gain a better understanding of them, implementation of a singly linked list and a doubly linked list in C++ proved to be helpful. In this example I only plan to illustrate the singly linked list. The text referenced above contains explanations of doubly linked lists and more functions than I plan to explain here.

First we'll create a header file to contain the LinkedList class.
"LinkedList.h" But, anything that makes sense to you should be fine.

Since linked lists are a collection of nodes, creating a Node struct, seemed like a good first step. Initializing them to NULL should help avoid strange values being pulled.

struct Node {
    int data = NULL;
    Node *next = NULL;
};

From here we can begin to create the outline of the class:

class LinkedList{
private:
    Node *head;
    Node *tail;
public:
    LinkedList() {  //constructor
        head = NULL;
        tail = NULL;
    };
    //functions will go here.
};

The head is a pointer to the first Node in the list while the tail is pointer to the last Node.  These two pointers have been set to NULL to avoid any complications random initialization might cause. I should also note that while "int" was used as the data type, this does not have to be the case. You can set the data type differently depending on the anticipated use.

There are a few different ways I've seen this structure implemented but, continuing with the example from Data Structures and Algorithms (DSA), there are 3 main functions to create. First is insertion. We always insert at the tail, unless it is the first node, then it is both the head and the tail. Because of this, we have the constant time complexity of $O(1)$.

void Add(int value) {
    Node *n = new Node;
    n->data = value;
    if (head == NULL) {
        //There are currently no nodes in this list
        this->head = n;
        this->tail = n;
    } else {
        //Add node to the end of the list
        this->tail->next = n;
        this->tail = n;
    }
};

Next is searching, which has a worst case time complexity of $O(n)$. The method is simple, we go through each node to check for the value we're searching for.

bool Contains(int value) {
    Node *n = this->head;
    while (n != NULL and n->data != value) {
        //Search through list for value. While loop breaks when value is found or tail node is reached
        n = n->next;
    }
    if (n == NULL) {
        //Since tail node points to nothing, we know we didn't find it
        return false;
    }
    return true;
};

Finally, removal from the list, which is also $O(n)$ because we check each node until we find the one we would like to remove. There are more cases to be aware of in this function. I have briefly explained them in the comments in the code.

bool Remove (int value) {
    //Check if the list is empty
    if (this->head == NULL) {
        return false;
    }
    Node *n = this->head;
        
    if (n->data == value) {
        if (this->head == tail) {
        //Check if node to remove is the only node
            this->head = NULL;
            this->tail = NULL;
        } else {
            //We must remove the head node
            head = head->next;
        }
        return true;
    }
        
    while (n->next != NULL and n->next->data != value) {
        //Search through list for value.
        n = n->next;
    }
    if (n->next != NULL) {
        if (n->next == tail) {
            //removing the tail node
            tail = n;
        } else {
            //removing some other internal node
            n->next = n->next->next;
        }
        return true;
    }
    return false;
}; 

This will be my stopping point for this data structure. It can be tested easily by including the header file in a cpp program and creating an instance of the LinkedList. For example:

#include "LinkedList.h"

using namespace std;

int main(int argc, const char * argv[])
{
    int a = 0;
    
    LinkedList exampleList;
    exampleList.Add(a);
    if (exampleList.Contains(a)) {
        cout << "true" << endl;
    }
    // else print false
    // other function tests
    // ...
    return 0;
};

DSA, continues with traversals which can be implemented fairly simply, as with the rest of these functions. One thing to note is that the reverse traversal has a time complexity of $O(n^2)$.This is because in order to find the previous node you must set a current node and search forwards from the head until you reach the current node's predecessor, thus we have an $O(n)$ complexity for the forward traversals multiplied by $n$, each node in the list, giving us $O(n^2)$.

If I think adding more functions/examples to this is necessary I will do so in the future.

Any questions, comments and/or suggestions are appreciated.

Note To Self

While I decided to create this blog for only one reason, I'm sure eventually more will arise.

Regardless, as I continue learning various things, I'm going to document them here; mostly for my own reference and to solidify my understanding. The subjects will mainly be math and computer science but, occasionally, I'll probably post about physics, biology, psychology, philosophy or some "interesting" topic.

Anyways, I plan to post soon, we'll see how that turns out...