Knapsack Problem
Algorithm · Core
Step through it: Dynamic Programming pattern
Step 1 of 6
Goal: how many ways can you climb 5 stairs taking 1 or 2 steps at a time? You reach stair 5 from stair 4 or stair 3, so ways(5) = ways(4) + ways(3). Working that out naively repeats the same sub-questions many times. Instead write each answer down once.
Step 2 of 6
Base cases: there is 1 way to be on stair 0 (do nothing) and 1 way to reach stair 1.
Step 3 of 6
Stair 2 is reached from stair 1 or stair 0.
Step 4 of 6
Stair 3 comes from stairs 2 and 1.
Step 5 of 6
Stair 4: 3 + 2 = 5. Stair 5: 5 + 3 = 8. Each answer reuses two stored answers, so n stairs take n steps instead of exponential time. Storing sub-answers like this is dynamic programming.
Step 6 of 6
Your turn.
The knapsack problem or rucksack problem is a problem in combinatorial optimization: Given a set of items, each with a weight and a value, determine the number of each item to include in a collection so that the total weight is less than or equal to a given limit and the total value is as large as possible.
It derives its name from the problem faced by someone who is constrained by a fixed-size knapsack and must fill it with the most valuable items.
Example of a one-dimensional (constraint) knapsack problem: which boxes should be chosen to maximize the amount of money while still keeping the overall weight under or equal to 15 kg?
Definition
0/1 knapsack problem
The most common problem being solved is the 0/1 knapsack problem,
which restricts the number xi of copies of each kind of item to zero or one.
Given a set of n items numbered from 1 up to n, each with a
weight wi and a value vi, along with a maximum weight
capacity W,
maximize
subject to and
Here xi represents the number of instances of item i to
include in the knapsack. Informally, the problem is to maximize
the sum of the values of the items in the knapsack so that the
sum of the weights is less than or equal to the knapsack's
capacity.
Bounded knapsack problem (BKP)
The bounded knapsack problem (BKP) removes the restriction
that there is only one of each item, but restricts the number
xi of copies of each kind of item to a maximum non-negative
integer value c:
maximize
subject to and
Unbounded knapsack problem (UKP)
The unbounded knapsack problem (UKP) places no upper bound
on the number of copies of each kind of item and can be
formulated as above except for that the only restriction
on xi is that it is a non-negative integer.
maximize
subject to and
Code
import MergeSort from '../../sorting/merge-sort/MergeSort';
export default class Knapsack {
/**
* @param {KnapsackItem[]} possibleItems
* @param {number} weightLimit
*/
constructor(possibleItems, weightLimit) {
this.selectedItems = [];
this.weightLimit = weightLimit;
this.possibleItems = possibleItems;
}
sortPossibleItemsByWeight() {
this.possibleItems = new MergeSort({
/**
* @var KnapsackItem itemA
* @var KnapsackItem itemB
*/
compareCallback: (itemA, itemB) => {
if (itemA.weight === itemB.weight) {
return 0;
}
return itemA.weight < itemB.weight ? -1 : 1;
},
}).sort(this.possibleItems);
}
sortPossibleItemsByValue() {
this.possibleItems = new MergeSort({
/**
* @var KnapsackItem itemA
* @var KnapsackItem itemB
*/
compareCallback: (itemA, itemB) => {
if (itemA.value === itemB.value) {
return 0;
}
return itemA.value > itemB.value ? -1 : 1;
},
}).sort(this.possibleItems);
}
sortPossibleItemsByValuePerWeightRatio() {
this.possibleItems = new MergeSort({
/**
* @var KnapsackItem itemA
* @var KnapsackItem itemB
*/
compareCallback: (itemA, itemB) => {
if (itemA.valuePerWeightRatio === itemB.valuePerWeightRatio) {
return 0;
}
return itemA.valuePerWeightRatio > itemB.valuePerWeightRatio ? -1 : 1;
},
}).sort(this.possibleItems);
}
// Solve 0/1 knapsack problem
// Dynamic Programming approach.
solveZeroOneKnapsackProblem() {
// We do two sorts because in case of equal weights but different values
// we need to take the most valuable items first.
this.sortPossibleItemsByValue();
this.sortPossibleItemsByWeight();
this.selectedItems = [];
// Create knapsack values matrix.
const numberOfRows = this.possibleItems.length;
const numberOfColumns = this.weightLimit;
const knapsackMatrix = Array(numberOfRows).fill(null).map(() => {
return Array(numberOfColumns + 1).fill(null);
});
// Fill the first column with zeros since it would mean that there is
// no items we can add to knapsack in case if weight limitation is zero.
for (let itemIndex = 0; itemIndex < this.possibleItems.length; itemIndex += 1) {
knapsackMatrix[itemIndex][0] = 0;
}
// Fill the first row with max possible values we would get by just adding
// or not adding the first item to the knapsack.
for (let weightIndex = 1; weightIndex <= this.weightLimit; weightIndex += 1) {
const itemIndex = 0;
const itemWeight = this.possibleItems[itemIndex].weight;
const itemValue = this.possibleItems[itemIndex].value;
knapsackMatrix[itemIndex][weightIndex] = itemWeight <= weightIndex ? itemValue : 0;
}
// Go through combinations of how we may add items to knapsack and
// define what weight/value we would receive using Dynamic Programming
// approach.
for (let itemIndex = 1; itemIndex < this.possibleItems.length; itemIndex += 1) {
for (let weightIndex = 1; weightIndex <= this.weightLimit; weightIndex += 1) {
const currentItemWeight = this.possibleItems[itemIndex].weight;
const currentItemValue = this.possibleItems[itemIndex].value;
if (currentItemWeight > weightIndex) {
// In case if item's weight is bigger then currently allowed weight
// then we can't add it to knapsack and the max possible value we can
// gain at the moment is the max value we got for previous item.
knapsackMatrix[itemIndex][weightIndex] = knapsackMatrix[itemIndex - 1][weightIndex];
} else {
// Else we need to consider the max value we can gain at this point by adding
// current value or just by keeping the previous item for current weight.
knapsackMatrix[itemIndex][weightIndex] = Math.max(
currentItemValue + knapsackMatrix[itemIndex - 1][weightIndex - currentItemWeight],
knapsackMatrix[itemIndex - 1][weightIndex],
);
}
}
}
// Now let's trace back the knapsack matrix to see what items we're going to add
// to the knapsack.
let itemIndex = this.possibleItems.length - 1;
let weightIndex = this.weightLimit;
while (itemIndex > 0) {
const currentItem = this.possibleItems[itemIndex];
const prevItem = this.possibleItems[itemIndex - 1];
// Check if matrix value came from top (from previous item).
// In this case this would mean that we need to include previous item
// to the list of selected items.
if (
knapsackMatrix[itemIndex][weightIndex]
&& knapsackMatrix[itemIndex][weightIndex] === knapsackMatrix[itemIndex - 1][weightIndex]
) {
// Check if there are several items with the same weight but with the different values.
// We need to add highest item in the matrix that is possible to get the highest value.
const prevSumValue = knapsackMatrix[itemIndex - 1][weightIndex];
const prevPrevSumValue = knapsackMatrix[itemIndex - 2][weightIndex];
if (
!prevSumValue
|| (prevSumValue && prevPrevSumValue !== prevSumValue)
) {
this.selectedItems.push(prevItem);
}
} else if (knapsackMatrix[itemIndex - 1][weightIndex - currentItem.weight]) {
this.selectedItems.push(prevItem);
weightIndex -= currentItem.weight;
}
itemIndex -= 1;
}
}
// Solve unbounded knapsack problem.
// Greedy approach.
solveUnboundedKnapsackProblem() {
this.sortPossibleItemsByValue();
this.sortPossibleItemsByValuePerWeightRatio();
for (let itemIndex = 0; itemIndex < this.possibleItems.length; itemIndex += 1) {
if (this.totalWeight < this.weightLimit) {
const currentItem = this.possibleItems[itemIndex];
// Detect how much of current items we can push to knapsack.
const availableWeight = this.weightLimit - this.totalWeight;
const maxPossibleItemsCount = Math.floor(availableWeight / currentItem.weight);
if (maxPossibleItemsCount > currentItem.itemsInStock) {
// If we have more items in stock then it is allowed to add
// let's add the maximum allowed number of them.
currentItem.quantity = currentItem.itemsInStock;
} else if (maxPossibleItemsCount) {
// In case if we don't have specified number of items in stock
// let's add only items we have in stock.
currentItem.quantity = maxPossibleItemsCount;
}
this.selectedItems.push(currentItem);
}
}
}
get totalValue() {
/** @var {KnapsackItem} item */
return this.selectedItems.reduce((accumulator, item) => {
return accumulator + item.totalValue;
}, 0);
}
get totalWeight() {
/** @var {KnapsackItem} item */
return this.selectedItems.reduce((accumulator, item) => {
return accumulator + item.totalWeight;
}, 0);
}
}export default class KnapsackItem {
/**
* @param {Object} itemSettings - knapsack item settings,
* @param {number} itemSettings.value - value of the item.
* @param {number} itemSettings.weight - weight of the item.
* @param {number} itemSettings.itemsInStock - how many items are available to be added.
*/
constructor({ value, weight, itemsInStock = 1 }) {
this.value = value;
this.weight = weight;
this.itemsInStock = itemsInStock;
// Actual number of items that is going to be added to knapsack.
this.quantity = 1;
}
get totalValue() {
return this.value * this.quantity;
}
get totalWeight() {
return this.weight * this.quantity;
}
// This coefficient shows how valuable the 1 unit of weight is
// for current item.
get valuePerWeightRatio() {
return this.value / this.weight;
}
toString() {
return `v${this.value} w${this.weight} x ${this.quantity}`;
}
}References
Practice
Dynamic Programming
- Climbing StairsEasy
- Fibonacci NumberEasy
- Min Cost Climbing StairsEasy
- Pascal's TriangleEasy
- House RobberMedium
- House Robber IIMedium
- Coin ChangeMedium
- Coin Change IIMedium
- Longest Increasing SubsequenceMedium
- Longest Common SubsequenceMedium
- Word BreakMedium
- Unique PathsMedium
- Unique Paths IIMedium
- Minimum Path SumMedium
- Jump GameMedium
- Partition Equal Subset SumMedium
- Target SumMedium
- Decode WaysMedium
- Maximum Product SubarrayMedium
- Longest Palindromic SubstringMedium
- Palindromic SubstringsMedium
- Best Time to Buy and Sell Stock with CooldownMedium
- TriangleMedium
- Maximal SquareMedium
- Edit DistanceHard
- Regular Expression MatchingHard
- Burst BalloonsHard
- Distinct SubsequencesHard
- Longest Valid ParenthesesHard
- Best Time to Buy and Sell Stock IVHard
Spotted a mistake or something unclear? Report it on GitHub.