Introduction
0/1 Knapsack pattern is based on the famous problem with the same name which is efficiently solved using Dynamic Programming (DP).
In this pattern, we will go through a set of problems to develop an understanding of DP. We will always start with a brute-force recursive solution to see the overlapping subproblems, i.e., realizing that we are solving the same problems repeatedly.
After the recursive solution, we will modify our algorithm to apply advanced techniques of Memoization and Bottom-Up Dynamic Programming to develop a complete understanding of this pattern.
Let’s jump onto our first problem.
*0/1 Knapsack (medium)
Design Gurus Educative.io
Introduction
Given the weights and profits of ‘N’ items, we are asked to put these items in a knapsack which has a capacity ‘C’. The goal is to get the maximum profit out of the items in the knapsack. Each item can only be selected once, as we don’t have multiple quantities of any item.
Let’s take the example of Merry, who wants to carry some fruits in the knapsack to get maximum profit. Here are the weights and profits of the fruits:
Items: { Apple, Orange, Banana, Melon }
Weights: { 2, 3, 1, 4 }
Profits: { 4, 5, 3, 7 }
Knapsack capacity: 5
Let’s try to put various combinations of fruits in the knapsack, such that their total weight is not more than 5:
Apple + Orange (total weight 5) => 9 profit
Apple + Banana (total weight 3) => 7 profit
Orange + Banana (total weight 4) => 8 profit
Banana + Melon (total weight 5) => 10 profit
This shows that Banana + Melon is the best combination as it gives us the maximum profit and the total weight does not exceed the capacity.
Problem Statement
Given two integer arrays to represent weights and profits of ‘N’ items, we need to find a subset of these items which will give us maximum profit such that their cumulative weight is not more than a given number ‘C’. Each item can only be selected once, which means either we put an item in the knapsack or we skip it.
Basic Solution
A basic brute-force solution could be to try all combinations of the given items (as we did above), allowing us to choose the one with maximum profit and a weight that doesn’t exceed ‘C’. Take the example of four items (A, B, C, and D), as shown in the diagram below. To try all the combinations, our algorithm will look like:
Here is a visual representation of our algorithm:
All green boxes have a total weight that is less than or equal to the capacity (7), and all the red ones have a weight that is more than 7. The best solution we have is with items [B, D] having a total profit of 22 and a total weight of 7.
Code
Here is the code for the brute-force solution:
1 | class Solution: |
Time and Space complexity
The time complexity of the above algorithm is exponential O(2^n), where ‘n’ represents the total number of items. This can also be confirmed from the above recursion tree. As we can see, we will have a total of ‘31’ recursive calls – calculated through (2^n) + (2^n) - 1, which is asymptotically equivalent to O(2^n).
The space complexity is O(n). This space will be used to store the recursion stack. Since the recursive algorithm works in a depth-first fashion, which means that we can’t have more than ‘n’ recursive calls on the call stack at any time.
Overlapping Sub-problems: Let’s visually draw the recursive calls to see if there are any overlapping sub-problems. As we can see, in each recursive call, profits and weights arrays remain constant, and only capacity and currentIndex change. For simplicity, let’s denote capacity with ‘c’ and currentIndex with ‘i’:
We can clearly see that ‘c:4, i=3’ has been called twice. Hence we have an overlapping sub-problems pattern. We can use Memoization to efficiently solve overlapping sub-problems.
Top-down Dynamic Programming with Memoization
Memoization is when we store the results of all the previously solved sub-problems and return the results from memory if we encounter a problem that has already been solved.
Since we have two changing values (capacity
and currentIndex
) in our recursive function knapsackRecursive()
, we can use a two-dimensional array to store the results of all the solved sub-problems. As mentioned above, we need to store results for every sub-array (i.e. for every possible index ‘i’) and for every possible capacity ‘c’.
Code
Here is the code with memoization (see changes in the highlighted lines):
1 | class Solution: |
Time and Space complexity
Since our memoization array dp[profits.length][capacity+1]
stores the results for all subproblems, we can conclude that we will not have more than N*C subproblems (where ‘N’ is the number of items and ‘C’ is the knapsack capacity). This means that our time complexity will be O(N\C)*.
The above algorithm will use O(N\C)* space for the memoization array. Other than that we will use O(N) space for the recursion call-stack. So the total space complexity will be O(N\C + N)*, which is asymptotically equivalent to O(N\C)*.
Bottom-up Dynamic Programming
Let’s try to populate our dp[][]
array from the above solution by working in a bottom-up fashion. Essentially, we want to find the maximum profit for every sub-array and for every possible capacity. This means that dp[i][c]
will represent the maximum knapsack profit for capacity ‘c’ calculated from the first ‘i’ items.
So, for each item at index ‘i’ (0 <= i < items.length) and capacity ‘c’ (0 <= c <= capacity), we have two options:
- Exclude the item at index ‘i’. In this case, we will take whatever profit we get from the sub-array excluding this item =>
dp[i-1][c]
- Include the item at index ‘i’ if its weight is not more than the capacity. In this case, we include its profit plus whatever profit we get from the remaining capacity and from remaining items =>
profit[i] + dp[i-1][c-weight[i]]
Finally, our optimal solution will be maximum of the above two values:
1 | dp[i][c] = max (dp[i-1][c], profit[i] + dp[i-1][c-weight[i]]) |
Let’s draw this visually and start with our base case of zero capacity:
Code
Here is the code for our bottom-up dynamic programming approach:
1 | class Solution: |
Time and Space complexity
The above solution has the time and space complexity of O(N\C)*, where ‘N’ represents total items and ‘C’ is the maximum capacity.
How can we find the selected items?
As we know, the final profit is at the bottom-right corner. Therefore, we will start from there to find the items that will be going into the knapsack.
As you remember, at every step we had two options: include an item or skip it. If we skip an item, we take the profit from the remaining items (i.e. from the cell right above it); if we include the item, then we jump to the remaining profit to find more items.
Let’s understand this from the above example:
- ‘22’ did not come from the top cell (which is 17); hence we must include the item at index ‘3’ (which is item ‘D’).
- Subtract the profit of item ‘D’ from ‘22’ to get the remaining profit ‘6’. We then jump to profit ‘6’ on the same row.
- ‘6’ came from the top cell, so we jump to row ‘2’.
- Again ‘6’ came from the top cell, so we jump to row ‘1’.
- ‘6’ is different than the top cell, so we must include this item (which is item ‘B’).
- Subtract the profit of ‘B’ from ‘6’ to get profit ‘0’. We then jump to profit ‘0’ on the same row. As soon as we hit zero remaining profit, we can finish our item search.
- Thus the items going into the knapsack are {B, D}.
Let’s write a function to print the set of items included in the knapsack.
1 | class Solution: |
Challenge
Can we further improve our bottom-up DP solution? Can you find an algorithm that has space complexity?
We only need one previous row to find the optimal solution!
1 | class Solution: |
The solution above is similar to the previous solution, the only difference is that we use i%2
instead if i
and (i-1)%2
instead if i-1
. This solution has a space complexity of O(2\C) = O(C)*, where ‘C’ is the maximum capacity of the knapsack.
This space optimization solution can also be implemented using a single array. It is a bit tricky, but the intuition is to use the same array for the previous and the next iteration!
If you see closely, we need two values from the previous iteration: dp[c]
and dp[c-weight[i]]
Since our inner loop is iterating over c:0-->capacity
, let’s see how this might affect our two required values:
- When we access
dp[c]
, it has not been overridden yet for the current iteration, so it should be fine. dp[c-weight[i]]
might be overridden if “weight[i] > 0”. Therefore we can’t use this value for the current iteration.
To solve the second case, we can change our inner loop to process in the reverse direction: c:capacity-->0
. This will ensure that whenever we change a value in dp[]
, we will not need it again in the current iteration.
Can you try writing this algorithm?
1 | class Solution: |
Equal Subset Sum Partition (medium)
416. Partition Equal Subset Sum Design Gurus Educative.io
Problem Statement
Given a set of positive numbers, find if we can partition it into two subsets such that the sum of elements in both subsets is equal.
Example 1:
1 | Input: {1, 2, 3, 4} |
Example 2:
1 | Input: {1, 1, 3, 4, 7} |
Example 3:
1 | Input: {2, 3, 4, 6} |
Constraints:
1 <= nums.length <= 200
1 <= nums[i] <= 100
Basic Solution
This problem follows the 0/1 Knapsack pattern. A basic brute-force solution could be to try all combinations of partitioning the given numbers into two sets to see if any pair of sets has an equal sum.
Assume that S
represents the total sum of all the given numbers. Then the two equal subsets must have a sum equal to S/2
. This essentially transforms our problem to: “Find a subset of the given numbers that has a total sum of S/2
“.
So our brute-force algorithm will look like:
Code
Here is the code for the brute-force solution:
1 | class Solution: |
Time and Space complexity
The time complexity of the above algorithm is exponential O(2^n), where ‘n’ represents the total number. The space complexity is O(n), which will be used to store the recursion stack.
Top-down Dynamic Programming with Memoization
We can use memoization to overcome the overlapping sub-problems. As stated in previous lessons, memoization is when we store the results of all the previously solved sub-problems so we can return the results from memory if we encounter a problem that has already been solved.
Since we need to store the results for every subset and for every possible sum, therefore we will be using a two-dimensional array to store the results of the solved sub-problems. The first dimension of the array will represent different subsets and the second dimension will represent different ‘sums’ that we can calculate from each subset. These two dimensions of the array can also be inferred from the two changing values (sum and currentIndex) in our recursive function canPartitionRecursive()
.
Code
Here is the code for Top-down DP with memoization:
1 | class Solution: |
Time and Space complexity
The above algorithm has the time and space complexity of O(N\S)*, where ‘N’ represents total numbers and ‘S’ is the total sum of all the numbers.
Bottom-up Dynamic Programming
Let’s try to populate our dp[][]
array from the above solution by working in a bottom-up fashion. Essentially, we want to find if we can make all possible sums with every subset. This means, dp[i][s]
will be ‘true’ if we can make the sum ‘s’ from the first ‘i’ numbers.
So, for each number at index ‘i’ (0 <= i < num.length) and sum ‘s’ (0 <= s <= S/2), we have two options:
- Exclude the number. In this case, we will see if we can get ‘s’ from the subset excluding this number:
dp[i-1][s]
- Include the number if its value is not more than ‘s’. In this case, we will see if we can find a subset to get the remaining sum:
dp[i-1][s-num[i]]
If either of the two above scenarios is true, we can find a subset of numbers with a sum equal to ‘s’.
Let’s start with our base case of zero capacity:
From the above visualization, we can clearly see that it is possible to partition the given set into two subsets with equal sums, as shown by bottom-right cell: dp[3][5] => T
Code
Here is the code for our bottom-up dynamic programming approach:
1 | class Solution: |
Time and Space complexity
The above solution the has time and space complexity of O(N\S)*, where ‘N’ represents total numbers and ‘S’ is the total sum of all the numbers.
Subset Sum (medium)
Similar | 416. Partition Equal Subset Sum Design Gurus Educative.io
Problem Statement
Given a set of positive numbers, determine if a subset exists whose sum is equal to a given number ‘S’.
Example 1:
1 | Input: {1, 2, 3, 7}, S=6 |
Example 2:
1 | Input: {1, 2, 7, 1, 5}, S=10 |
Example 3:
1 | Input: {1, 3, 4, 8}, S=6 |
Constraints:
1 <= num.length <= 200
1 <= num[i] <= 100
Basic Solution
This problem follows the 0/1 Knapsack pattern and is quite similar to Equal Subset Sum Partition. A basic brute-force solution could be to try all subsets of the given numbers to see if any set has a sum equal to ‘S’.
So our brute-force algorithm will look like:
Since this problem is quite similar to Equal Subset Sum Partition, let’s jump directly to the bottom-up dynamic programming solution.
Bottom-up Dynamic Programming
We’ll try to find if we can make all possible sums with every subset to populate the array dp[TotalNumbers][S+1]
.
For every possible sum ‘s’ (where 0 <= s <= S), we have two options:
- Exclude the number. In this case, we will see if we can get the sum ‘s’ from the subset excluding this number =>
dp[index-1][s]
- Include the number if its value is not more than ‘s’. In this case, we will see if we can find a subset to get the remaining sum =>
dp[index-1][s-num[index]]
If either of the above two scenarios returns true, we can find a subset with a sum equal to ‘s’.
Let’s draw this visually, with the example input {1, 2, 3, 7}, and start with our base case of size zero:
Code
Here is the code for our bottom-up dynamic programming approach:
1 | class Solution: |
Time and Space complexity
The above solution has the time and space complexity of O(N\S)*, where ‘N’ represents total numbers and ‘S’ is the required sum.
Challenge
Can we improve our bottom-up DP solution even further? Can you find an algorithm that has O(S) space complexity?
Hint
Similar to the space optimized solution for 0/1 Knapsack
1 | class Solution: |
Minimum Subset Sum Difference (hard)
leetcode 2035 但是没有限制positive num,不太会了
Similar | 2035. Partition Array Into Two Arrays to Minimize Sum Difference Design Gurus Educative.io
Problem Statement
Given a set of positive numbers, partition the set into two subsets with minimum difference between their subset sums.
Example 1:
1 | Input: {1, 2, 3, 9} |
Example 2:
1 | Input: {1, 2, 7, 1, 5} |
Example 3:
1 | Input: {1, 3, 100, 4} |
Constraints:
1 <= num.length <= 200
1 <= num[i] <= 100
Basic Solution
This problem follows the 0/1 Knapsack pattern and can be converted into a Subset Sum problem.
Let’s assume S1 and S2 are the two desired subsets. A basic brute-force solution could be to try adding each element either in S1 or S2 in order to find the combination that gives the minimum sum difference between the two sets.
So our brute-force algorithm will look like:
Code
Here is the code for the brute-force solution:
1 | class Solution: |
Time and Space complexity
Because of the two recursive calls, the time complexity of the above algorithm is exponential O(2^n), where ‘n’ represents the total number. The space complexity is O(n) which is used to store the recursion stack.
Top-down Dynamic Programming with Memoization
We can use memoization to overcome the overlapping sub-problems.
We will be using a two-dimensional array to store the results of the solved sub-problems. We can uniquely identify a sub-problem from ‘currentIndex’ and ‘Sum1’ as ‘Sum2’ will always be the sum of the remaining numbers.
Code
Here is the code:
1 | class Solution: |
Bottom-up Dynamic Programming
Let’s assume ‘S’ represents the total sum of all the numbers. So, in this problem, we are trying to find a subset whose sum is as close to ‘S/2’ as possible, because if we can partition the given set into two subsets of an equal sum, we get the minimum difference, i.e. zero. This transforms our problem to Subset Sum, where we try to find a subset whose sum is equal to a given number— ‘S/2’ in our case. If we can’t find such a subset, then we will take the subset which has the sum closest to ‘S/2’. This is easily possible, as we will be calculating all possible sums with every subset.
Essentially, we need to calculate all the possible sums up to ‘S/2’ for all numbers. So how can we populate the array db[TotalNumbers][S/2+1]
in the bottom-up fashion?
For every possible sum ‘s’ (where 0 <= s <= S/2), we have two options:
- Exclude the number. In this case, we will see if we can get the sum ‘s’ from the subset excluding this number =>
dp[index-1][s]
- Include the number if its value is not more than ‘s’. In this case, we will see if we can find a subset to get the remaining sum =>
dp[index-1][s-num[index]]
If either of the two above scenarios is true, we can find a subset with a sum equal to ‘s’. We should dig into this before we can learn how to find the closest subset.
Let’s draw this visually, with the example input {1, 2, 3, 9}. Since the total sum is ‘15’, we will try to find a subset whose sum is equal to the half of it, i.e. ‘7’.
The above visualization tells us that it is not possible to find a subset whose sum is equal to ‘7’. So what is the closest subset we can find? We can find the subset if we start moving backwards in the last row from the bottom right corner to find the first ‘T’. The first “T” in the diagram above is the sum ‘6’, which means that we can find a subset whose sum is equal to ‘6’. This means the other set will have a sum of ‘9’ and the minimum difference will be ‘3’.
Code
Here is the code for our bottom-up dynamic programming approach:
1 | class Solution: |
Time and Space complexity
The above solution has the time and space complexity of O(N\S)*, where ‘N’ represents total numbers and ‘S’ is the total sum of all the numbers.
Problem Challenge 1
Similar | Top Interview 150 | 39. Combination Sum Design Gurus Educative.io
Count of Subset Sum (hard)
Given a set of positive numbers, find the total number of subsets whose sum is equal to a given number ‘S’.
Example 1:
1 | Input: {1, 1, 2, 3}, S=4 |
Example 2:
1 | Input: {1, 2, 7, 1, 5}, S=9 |
Constraints:
1 <= num.length <= 20
0 <= num[i] <= 1000
0 <= sum(num[i]) <= 1000
-1000 <= S <= 1000
Basic Solution
This problem follows the 0/1 Knapsack pattern and is quite similar to Subset Sum. The only difference in this problem is that we need to count the number of subsets, whereas in Subset Sum we only wanted to know if a subset with the given sum existed.
A basic brute-force solution could be to try all subsets of the given numbers to count the subsets that have a sum equal to ‘S’. So our brute-force algorithm will look like:
Code
Here is the code for the brute-force solution:
1 | class Solution: |
Time and Space complexity
The time complexity of the above algorithm is exponential O(2^n), where ‘n’ represents the total number. The space complexity is O(n), this memory is used to store the recursion stack.
Top-down Dynamic Programming with Memoization
We can use memoization to overcome the overlapping sub-problems. We will be using a two-dimensional array to store the results of solved sub-problems. As mentioned above, we need to store results for every subset and for every possible sum.
Code
Here is the code:
1 | class Solution: |
Bottom-up Dynamic Programming
We will try to find if we can make all possible sums with every subset to populate the array db[TotalNumbers][S+1
].
So, at every step we have two options:
- Exclude the number. Count all the subsets without the given number up to the given sum =>
dp[index-1][sum]
- Include the number if its value is not more than the ‘sum’. In this case, we will count all the subsets to get the remaining sum =>
dp[index-1][sum-num[index]]
To find the total sets, we will add both of the above two values:
1 | dp[index][sum] = dp[index-1][sum] + dp[index-1][sum-num[index]]) |
Let’s start with our base case of size zero:
Code
Here is the code for our bottom-up dynamic programming approach:
1 | class Solution: |
Time and Space complexity
The above solution has the time and space complexity of O(N\S)*, where ‘N’ represents total numbers and ‘S’ is the desired sum.
Challenge
Can we improve our bottom-up DP solution even further? Can you find an algorithm that has O(S) space complexity?
1 | class Solution: |
Problem Challenge 2
leetcode 494, leetcode里有0,不太会,尝试了两次都失败了
Similar | 494. Target Sum Design Gurus Educative.io
Target Sum (hard)
You are given a set of positive numbers and a target sum ‘S’. Each number should be assigned either a ‘+’ or ‘-’ sign. We need to find the total ways to assign symbols to make the sum of the numbers equal to the target ‘S’.
Example 1:
1 | Input: {1, 1, 2, 3}, S=1 |
Example 2:
1 | Input: {1, 2, 7, 1}, S=9 |
Constraints:
1 <= num.length <= 20
0 <= num[i] <= 1000
0 <= sum(num[i]) <= 1000
-1000 <= S <= 1000
Solution
This problem follows the 0/1 Knapsack pattern and can be converted into Count of Subset Sum. Let’s dig into this.
We are asked to find two subsets of the given numbers whose difference is equal to the given target ‘S’. Take the first example above. As we saw, one solution is {+1-1-2+3}. So, the two subsets we are asked to find are {1, 3} & {1, 2} because,
1 | (1 + 3) - (1 + 2 ) = 1 |
Now, let’s say ‘Sum(s1)’ denotes the total sum of set ‘s1’, and ‘Sum(s2)’ denotes the total sum of set ‘s2’. So the required equation is:
1 | Sum(s1) - Sum(s2) = S |
This equation can be reduced to the subset sum problem. Let’s assume that ‘Sum(num)’ denotes the total sum of all the numbers, therefore:
1 | Sum(s1) + Sum(s2) = Sum(num) |
Let’s add the above two equations:
1 | => Sum(s1) - Sum(s2) + Sum(s1) + Sum(s2) = S + Sum(num) |
Which means that one of the set ‘s1’ has a sum equal to (S + Sum(num)) / 2
. This essentially converts our problem to: “Find the count of subsets of the given numbers whose sum is equal to (S + Sum(num)) / 2
“
Code
Let’s take the dynamic programming code of Count of Subset Sum and extend it to solve this problem:
1 | class Solution: |
Time and Space complexity
The above solution has time and space complexity of O(N*S), where ‘N’ represents total numbers and ‘S’ is the desired sum.
We can further improve the solution to use only O(S) space.
Space Optimized Solution
Here is the code for the space-optimized solution, using only a single array:
1 | class Solution: |