Articles not classified by category

[CMU15445] Lock and transaction isolation level

• Author: rin
• Time: August 16, 2023
• Classification: Unclassified
• comment

<trans data-src="「基于锁的并发控制」与「多版本并发控制」是实现事务隔离级别的两种不同思路。">"Lock based concurrency control" and "Multi version concurrency control" are two different ideas to achieve transaction isolation level.</trans> <trans data-src="在两阶段锁中，事务分为获取锁的**增长阶段**和释放锁的**收缩阶段**。">In two-phase locks, transactions are divided into the * * growth phase * * of acquiring locks and the * * contraction phase * * of releasing locks.</trans> <trans data-src="在不同的阶段中通过设置不同的规则和限制可以实现不同的隔离级别。">Different isolation levels can be achieved by setting different rules and restrictions in different stages.</trans> <trans data-src="例如：">For example:</trans> <trans data-src="1. 读取数据时是否需要锁？">1. Is a lock required when reading data?</trans> <trans data-src="需要什么类型的锁？">What type of lock is required?</trans> <trans data-src="2. 什么时候释放锁？">2. When will the lock be released?</trans> <trans data-src="不同的隔离级别下，事务修改数据时都需要X锁，并在事务结束时释放。">Under different isolation levels, X locks are required when a transaction modifies data and are released at the end of the transaction.</trans> <trans data-src="这将保证两个写事务之间不会产生**脏写**的问题，这种问题在数据库中是不允许的。">This will ensure that the * * dirty write * * problem does not occur between two write transactions. This problem is not allowed in the database.</trans> <trans data-src="因此在不同隔离级别下都可能会用到X锁，但S锁不一定。">Therefore, the X lock may be used under different isolation levels, but the S lock is not necessarily used.</trans> <trans data-src="---">---</trans> <trans data-src="**READ UNCOMMITTED**">**READ UNCOMMITTED**</trans> <trans data-src="1. 事务读数据时不需要S锁，只需要在写数据时获取X锁。">1. The transaction does not need the S lock when reading data, but only needs to obtain the X lock when writing data.</trans> <trans data-src="这样当前事务的读操作就不会被其他事务的X锁阻塞；">In this way, the read operation of the current transaction will not be blocked by the X lock of other transactions;</trans> <trans data-src="2. 事务结束时释放X锁。">2. Release the X lock when the transaction ends.</trans> <trans data-src="释放X锁事务则进入收缩阶段。">Releasing the X lock transaction will enter the shrink phase.</trans> <trans data-src="由于读操作不会被阻塞，可以随时读到别的事务修改的数据，因此存在**脏读**问题。">Since the read operation will not be blocked and the data modified by other transactions can be read at any time, the problem of * * dirty read * * exists.</trans> <trans data-src="---">---</trans> <trans data-src="**READ COMMITTED**">**READ COMMITTED**</trans> <trans data-src="1. 当前事务读数据时需要S锁，以阻塞之后其他事务对数据的修改。">1. When the current transaction reads data, it needs an S lock to block the modification of data by other transactions.</trans> <trans data-src="同样，当其他事务先获取了X锁并修改数据时，当前事务对S锁的获取将被阻塞，直到其他事务的X锁释放才能读到数据。">Similarly, when other transactions first acquire the X lock and modify the data, the acquisition of the S lock by the current transaction will be blocked, and the data cannot be read until the X lock of other transactions is released.</trans> <trans data-src="2. 事务结束时释放X锁，释放X锁事务则进入收缩阶段。">2. Release the X lock when the transaction ends, and the X lock release transaction enters the shrink phase.</trans> <trans data-src="释放S锁不改变事务的阶段，因此可以随时释放（？）">Releasing the S lock does not change the transaction phase, so it can be released at any time (?)</trans> <trans data-src="READ COMMITTED解决了脏读问题：S锁会被X锁阻塞+事务结束时才释放X锁=只能读到提交/回滚的数据。">READ COMMITTED solves the dirty read problem: the S lock will be blocked by the X lock+the X lock will be released at the end of the transaction=only committed/rolled back data can be read.</trans> <trans data-src="但是没有解决不可重复读的问题：当前事务读完数据之后可以释放S锁，这时其他事务可以修改相应的数据并提交，当前事务再次读取时将得到不同的数据，即**不可重复读**问题。">However, the problem of non repeatable reading has not been solved: the S lock can be released after the current transaction has read the data. At this time, other transactions can modify the corresponding data and submit it. When the current transaction reads again, it will get different data, that is, the problem of * * non repeatable reading * *.</trans> <trans data-src="---">---</trans> <trans data-src="**REPEATABLE READ**">**REPEATABLE READ**</trans> <trans data-src="1. 与READ COMMITTED相同">1. Same as READ COMMITTED</trans> <trans data-src="2. 事务结束时释放X锁与S锁。">2. Release the X lock and the S lock at the end of the transaction.</trans> <trans data-src="事务进入收缩阶段。">The transaction enters the shrink phase.</trans> <trans data-src="由于S锁也会被保留到事务结束，因此其他事务无法修改相应的数据，当前事务中多次读取得到的也是同样的数据。">Since the S lock is also retained until the end of the transaction, other transactions cannot modify the corresponding data. The same data is obtained from multiple reads in the current transaction.</trans> <trans data-src="| | S锁 | X锁 |">|| S lock | X lock|</trans> <trans data-src="| --- | ------ | --- |">| --- | ------ | --- |</trans> <trans data-src="| RU | N/A | 结束时释放 |">|RU | N/A | Release at the end|</trans> <trans data-src="| RC | 读后释放 | 结束时释放 |">|RC | Release after reading | Release at the end|</trans> <trans data-src="| RR | 结束时释放 | 结束时释放 |">|RR | Release at the end | Release at the end|</trans>

[CMU15445] bustub 2023 Project3

• Author: rin
• Time: May 20, 2023
• Classification: Unclassified
• comment

<trans data-src="1. `ExecutorContext *exec_ctx` 是什么？">1. What is' ExecutorContext * exec_ctx '?</trans> <trans data-src="2. 如何根据plan中的table_name找到要seq_scan的table？">2. How to find the table to seq_scan according to the table_name in the plan?</trans> <trans data-src="（or table_heap?）">（or table_heap?）</trans> <trans data-src="1. 查找到catalog.h中有关于table_name的信息">1. Find the information about table_name in catalog. h</trans> <trans data-src="2. The Catalog is a non-persistent catalog that is designed for use by executors within the DBMS execution engine. ">2. The Catalog is a non-persistent catalog that is designed for use by executors within the DBMS execution engine. </trans> <trans data-src="It handles table creation, **table lookup**, index creation, and index lookup.">It handles table creation, **table lookup**, index creation, and index lookup.</trans> <trans data-src="3. 查到`ExecutorContext`中包含了catalog，所以应该可以从`exec_ctx_`中获得table">3. It is found that the 'ExecutorContext' contains a catalog, so you can get the table from 'exec_ctx_'</trans> <trans data-src="## insert">## insert</trans> <trans data-src="3. 参考projection_executor写了insert_exector的构造函数及init、next">3. The constructor, init and next of insert_exector are written by referring to project_exector</trans> <trans data-src="1. 使用InsertTuple插入时，需要TupleMeta和Tuple，后者可以从child_executor->Next中获取，前者呢？——">1. When inserting with InsertTuple, TupleMeta and Tuple are required. The latter can be obtained from child_executor ->Next. What about the former——</trans> <trans data-src="只需手动设置TupleMeta的那三个字段">Just manually set the three fields of TupleMeta</trans> <trans data-src="2. 需要通过\*tuple返回插入的数量，如何构造这个Tuple？">2. The number of inserts needs to be returned through * tuple. How to construct this tuple?</trans> <trans data-src="构造tuple时也需要对应的schema，这个从何而来？——">When constructing tuples, the corresponding schema is also required. Where does this come from——</trans> <trans data-src="tuple需要一个vector\<Value> ，这个Value可以通过ValueFactory::GetIntegerValue()构造，而schema已经由planner传过来了，即output_schema，代表这个sql输出的schema是什么样">The tuple needs a vector <Value>, which can be constructed through ValueFactory:: GetIntegerValue(), and the schema has been transferred from the planner, that is, output_schema, which represents the schema output by the sql</trans> <trans data-src="3. 按照迭代器模型（volcano模型）的思想，我觉得在单次调用insert_executor->Next()的时候，在该executor内部应该也只应该调用一次child_executor->Next()，直到它返回false时我再返回false。">3. According to the idea of the iterator model (volcano model), I think that when insert_executor ->Next() is called once, child_executor ->Next() should be called only once inside the executor, and I will return false until it returns false.</trans> <trans data-src="但是在调用者（execution_engine.h:PollExecutor()）的代码中，它每一次执行我的Next时，都会往最终的result_set中插入一次tuple，这会导致该语句输出的结果中有多个row">But in the code of the caller (execution_engine. h: PollExecutor()), every time it executes My Next, it will insert a tuple into the final result_set, which will result in multiple rows in the output result of the statement</trans> <trans data-src="```">```</trans> <trans data-src="<main>:8"><main>:8</trans> <trans data-src="insert into t1 values (0, 'Emoji', 10), (1, 'EmojiEmoji', 11), (2, 'EmojiEmojiEmoji', 12), (3, 'EmojiEmojiEmojiEmoji', 13), (4, 'EmojiEmojiEmojiEmojiEmoji', 14);">insert into t1 values (0, 'Emoji', 10), (1, 'EmojiEmoji', 11), (2, 'EmojiEmojiEmoji', 12), (3, 'EmojiEmojiEmojiEmoji', 13), (4, 'EmojiEmojiEmojiEmojiEmoji', 14);</trans> <trans data-src="--- YOUR RESULT ---">--- YOUR RESULT ---</trans> <trans data-src="1 ">one</trans> <trans data-src="2 ">two</trans> <trans data-src="3 ">three</trans> <trans data-src="4 ">four</trans> <trans data-src="5 ">five</trans> <trans data-src="--- EXPECTED RESULT ---">--- EXPECTED RESULT ---</trans> <trans data-src="5">five</trans> <trans data-src="```">```</trans> <trans data-src="因此我一次性在Next()中使用循环将child_executor中产生的所有tuples处理完">So I use a loop in Next() to process all tuples generated in child_executor</trans> <trans data-src="---">---</trans> <trans data-src="在insert后，还需要插入index。">After inserting, you need to insert the index.</trans> <trans data-src="从`catalog->GetTableIndexes()`可以获得该table所有的index。">All indexes of the table can be obtained from 'catalog ->GetTableIndexes()'.</trans> <trans data-src="但是每个index覆盖的字段也不一样，无脑将新插入的这个tuple原样往index里插会导致segmentfault，要如何根据特定index schema去insert新的index？">However, the fields covered by each index are different, and mindless insertion of the newly inserted tuple into the index will result in a segmentfault. How do you insert a new index according to a specific index schema?</trans> <trans data-src="可以根据`index_info->key_scheme_`获得这个index的scheme，然后又根据这个table的scheme进行对应，选择需要的column生成key tuple。">The scheme of this index can be obtained according to 'index_info ->key_scheme_', and then corresponding to the scheme of this table. Select the required column to generate a key tuple.</trans> <trans data-src="在tuple.h中已经实现了`Tuple::KeyFromTuple()`可以直接使用。">'Tuple:: KeyFromTuple()' has been implemented in tuple. h and can be used directly.</trans> <trans data-src="## delete">## delete</trans> <trans data-src="4. 在delete_executor中同样使用循环处理所有的tuples，在TupleMeta中修改删除标记。">4. Repeat all tuples in delete_executor, and modify the deletion mark in TupleMeta.</trans> <trans data-src="在seq_search_executor->Next()中对于已被标记删除的tuple，应该忽略它，而不是立刻返回false，因为可能还有别的tuples要输出">In seq_search_executor ->Next(), the tuple that has been marked for deletion should be ignored instead of returning false immediately, because there may be other tuples to output</trans> <trans data-src="## update">## update</trans> <trans data-src="6. 在update_executor中，检查测试样例`explain update t1 set v3 = 445 where v1 >= 3;`">6. In update_executor, check the test sample ` explain update t1 set v3=445 where v1>=3`</trans> <trans data-src="的结果为">The result of is</trans> <trans data-src="```">```</trans> <trans data-src="=== BINDER ===">=== BINDER ===</trans> <trans data-src="BoundUpdate {">BoundUpdate {</trans> <trans data-src="table=BoundBaseTableRef { table=t1, oid=22 },">table=BoundBaseTableRef { table=t1, oid=22 },</trans> <trans data-src="filter_expr=(t1.v1>=3),">filter_expr=(t1.v1>=3),</trans> <trans data-src="target_expr=[(t1.v3, 445)],">target_expr=[(t1.v3, 445)],</trans> <trans data-src="}">}</trans> <trans data-src="=== PLANNER ===">=== PLANNER ===</trans> <trans data-src="Update { table_oid=22, target_exprs=[#0.0, #0.1, 445] } | (__bustub_internal.update_rows:INTEGER)">Update { table_oid=22, target_exprs=[#0.0, #0.1, 445] } | (__bustub_internal.update_rows:INTEGER)</trans> <trans data-src="Filter { predicate=(#0.0>=3) } | (t1.v1:INTEGER, t1.v2:VARCHAR, t1.v3:INTEGER)">Filter { predicate=(#0.0>=3) } | (t1.v1:INTEGER, t1.v2:VARCHAR, t1.v3:INTEGER)</trans> <trans data-src="SeqScan { table=t1 } | (t1.v1:INTEGER, t1.v2:VARCHAR, t1.v3:INTEGER)">SeqScan { table=t1 } | (t1.v1:INTEGER, t1.v2:VARCHAR, t1.v3:INTEGER)</trans> <trans data-src="=== OPTIMIZER ===">=== OPTIMIZER ===</trans> <trans data-src="Update { table_oid=22, target_exprs=[#0.0, #0.1, 445] } | (__bustub_internal.update_rows:INTEGER)">Update { table_oid=22, target_exprs=[#0.0, #0.1, 445] } | (__bustub_internal.update_rows:INTEGER)</trans> <trans data-src="SeqScan { table=t1, filter=(#0.0>=3) } | (t1.v1:INTEGER, t1.v2:VARCHAR, t1.v3:INTEGER)">SeqScan { table=t1, filter=(#0.0>=3) } | (t1.v1:INTEGER, t1.v2:VARCHAR, t1.v3:INTEGER)</trans> <trans data-src="```">```</trans> <trans data-src="对应的，代码中的child_executor即Filter，向update_executor提供需要更新的tuples。">Correspondingly, the child_executor in the code is the Filter, which provides update_executor with tuples that need to be updated.</trans> <trans data-src="然后新值存储在plan中的plan_->target_expressions_中。">Then the new value is stored in plan_ ->target_expressions_ in the plan.</trans> <trans data-src="即`target_exprs=[#0.0, #0.1, 445]`">That is, ` target_exprs=[# 0.0, # 0.1, 445]`</trans> <trans data-src="经过plan_->target_expressions_->Evaluate()后可以得到各列实际的值：`[3,EmojiEmojiEmojiEmoji,445]`">After plan_ ->target_expressions_ ->Evaluate(), the actual values of each column can be obtained: ` [3, EmojiEmojiEmojiEmojiEmoji, 445]`</trans> <trans data-src="```">```</trans> <trans data-src="update t1 set v3 = 445 where v1 >= 3;">update t1 set v3 = 445 where v1 >= 3;</trans> <trans data-src="2023-05-19 18:47:41 [update_executor.cpp:40:Next] DEBUG - Update from (3, EmojiEmojiEmojiEmoji, 13) Tuple size is 41">2023-05-19 18:47:41 [update_executor.cpp:40:Next] DEBUG - Update from (3, EmojiEmojiEmojiEmoji, 13) Tuple size is 41</trans> <trans data-src="2023-05-19 18:47:41 [update_executor.cpp:41:Next] DEBUG - To">2023-05-19 18:47:41 [update_executor.cpp:41:Next] DEBUG - To</trans> <trans data-src="2023-05-19 18:47:41 [update_executor.cpp:43:Next] DEBUG - Type: 4 Val: #0.0 Evaluated Val:3">2023-05-19 18:47:41 [update_executor.cpp:43:Next] DEBUG - Type: 4 Val: #0.0 Evaluated Val:3</trans> <trans data-src="2023-05-19 18:47:41 [update_executor.cpp:43:Next] DEBUG - Type: 7 Val: #0.1 Evaluated Val:EmojiEmojiEmojiEmoji">2023-05-19 18:47:41 [update_executor.cpp:43:Next] DEBUG - Type: 7 Val: #0.1 Evaluated Val:EmojiEmojiEmojiEmoji</trans> <trans data-src="2023-05-19 18:47:41 [update_executor.cpp:43:Next] DEBUG - Type: 4 Val: 445 Evaluated Val:445">2023-05-19 18:47:41 [update_executor.cpp:43:Next] DEBUG - Type: 4 Val: 445 Evaluated Val:445</trans> <trans data-src="```">```</trans> <trans data-src="## index_scan">## index_scan</trans> <trans data-src="在初始化BPlusTreeIndexIteratorForTwoIntegerColumn（IndexIterator）时，出现链接错误，找不到IndexIterator，即使对特定的那几个类型进行了explicit instantiation也不行，原因仅仅是因为这里使用了无参的构造函数，但是在index_iterator.h中没有定义，补上定义即可。">When initializing the BPlusTreeIndexIterator ForTwoIntegerColumn (IndexIterator), a link error occurred, and the IndexIterator could not be found, even if explicit instantiation was performed for specific types. The reason is that the construction function without parameters is used here, but it is not defined in index_iterator. h, so it is sufficient to supplement the definition.</trans> <trans data-src="## agg">## agg</trans> <trans data-src="首先根据注释，给aggregation_executor.h添加成员`SimpleAggregationHashTable aht_;`">First, add members to aggregation_executor. h 'SimpleAggregationHashTable aht_`</trans> <trans data-src="和`SimpleAggregationHashTable::Iterator aht_iterator_;`，">And 'SimpleAggregationHashTable:: Iterator aht_iterator_;',</trans> <trans data-src="并在构造函数上初始化。">And initialize on the constructor.</trans> <trans data-src="1. `SimpleAggregationHashTable`是什么？——">1. What is' SimpleAggregationHashTable '——</trans> <trans data-src="是一个`AggregateKey->AggregateValue`的hashtable">It is a hashtable of 'AggregateKey ->AggregateValue'</trans> <trans data-src="2. `AggregateKey`代表“ a key in an aggregation operation”，包含一个`std::vector<Value> group_bys_;`。">2. 'AggregateKey' stands for "a key in an aggregation operation", including a 'std:: vector<Value>group_bys_;'.</trans> <trans data-src="这个key应该与aggregate的columns有关。">This key should be related to the columns of aggregate.</trans> <trans data-src="3. `AggregateValue`里有一个`std::vector<Value> aggregates_;`，">3. There is a 'std:: vector<Value>aggregates_;' in 'AggregateValue',</trans> <trans data-src="应该是aggregate后的值。">It should be the value after aggregate.</trans> <trans data-src="1. 但是aggregate 函数如`CountStarAggregate, CountAggregate, SumAggregate, MinAggregate, MaxAggregate`这些也只产生一个值，或许vector里存了多个group产生的值？——">1. But aggregate functions such as' CountStarAggregate, CountAggregate, SumAggregate, MinAggregate, MaxAggregate 'also produce only one value. Maybe there are values generated by multiple groups stored in vector——</trans> <trans data-src="不是，vector里存的是不同聚集函数产生的值，第i个元素值代表`agg_types_[i]`这个类型的聚集函数">No, what is stored in vector is the value generated by different aggregation functions. The value of the ith element represents the 'agg_types_ [i]' aggregation function of this type</trans> <trans data-src="2. 在`aggregation_executor->Next()`中，像其他task一样，从child获取一个tuple然后进行处理。">2. In 'aggregation_executor ->Next()', like other tasks, get a tuple from the child and process it.</trans> <trans data-src="处理过程大概是构造AggregateKey和AggregateValue然后添加到某处">The process is to construct AggregateKey and AggregateValue and then add them somewhere</trans> <trans data-src="1. 查看`aggregation_executor.h`，其中有`MakeAggregateKey(),MakeAggregateValue(),aht_.">1. View 'aggregation_executor. h', where 'MakeAggregateKey(), MakeAggregateValue(), aht_</trans> <trans data-src="InsertCombine`几个函数。">InsertCombine ` Several functions.</trans> <trans data-src="构造Key的这个函数从plan_中获得**group_by_的expr**，使用evaluate求出keys；">The function to construct a key obtains * * expr * * of group_by_ from plan_, and uses evaluate to calculate keys;</trans> <trans data-src="构造Value的函数从plan_中获得**aggregate的expr**，使用evalutae求出values。">The function to construct Value obtains * * aggregate expr * * from plan_, and uses evalutae to calculate values.</trans> <trans data-src="2. 以第二个样例`insert into t1 values (-99999), (99999), (0), (1), (2), (3); ">2. Use the second example 'insert into t1 values (- 999999), (99999), (0), (1), (2), (3);</trans> <trans data-src="select count(*), min(v1), max(v1), count(v1), sum(v1) from t1;`">select count(*), min(v1), max(v1), count(v1), sum(v1) from t1;`</trans> <trans data-src="为例，第一次Next返回-99999这个tuple，MakeAggregateValue会用`count,min,max,count,sum`这五个函数产生`1,-99999,-99999,-99999,-99999`这五个值">For example, the first time Next returns the tuple of - 99999, MakeAggregateValue will use the five functions of 'count, min, max, count, sum' to generate the five values of '1, - 99999, - 99999, - 99999, - 99999, - 99999'</trans> <trans data-src="3. 在`CombineAggregateValues()`中，对于这五个值，就需要根据对于`agg_expr`的类型(`agg_types_[i]`)进行聚合处理。">3. In 'CombineAggregateValues()', these five values need to be aggregated according to the type of 'agg_expr' ('agg_types_ [i]').</trans> <trans data-src="4. 注意区分`count(*), count(column)`，前者统计总行数，后者统计该列中非空的数量。">4. Pay attention to distinguish between 'count (*)' and 'count (column)'. The former counts the total number of rows, and the latter counts the number of non empty columns in the column.</trans> <trans data-src="对于空表，前者为0；">For empty tables, the former is 0;</trans> <trans data-src="对于空列或者该全为null的列，后者为null。">For the empty column or the all null column, the latter is null.</trans> <trans data-src="对于其他聚集函数来说也是，input是null时跳过处理">For other aggregate functions, skip processing when input is null</trans> <trans data-src="5. executor的返回值是group_by columns + aggregation columns">5. The return value of executor is group_by columns+aggregation columns</trans> <trans data-src="> The output schema consists of the group-by columns followed by the aggregation columns.">> The output schema consists of the group-by columns followed by the aggregation columns.</trans> <trans data-src="如果我还select了其他column，但是没有orderby它，会怎样？——">What if I also select other columns, but do not have orderby——</trans> <trans data-src="这种情况可以是不允许的，参考MySQL的`only_full_group_by`。">This situation is not allowed. Please refer to MySQL's' only_full_group_by '.</trans> <trans data-src="因此不用处理非聚集的值和非groupby的值">So you don't have to deal with non aggregated values and non groupby values</trans> <trans data-src="例如，`__mock_table_1.colA:INTEGER`是groupby columns，`agg#0:INTEGER, agg#1:INTEGER, agg#2:INTEGER`是aggregate column">For example, '__mock_table_1. colA: INTEGER' is group by columns, 'agg # 0: INTEGER, agg # 1: INTEGER, agg # 2: INTEGER' is aggregate column</trans> <trans data-src="```">```</trans> <trans data-src="EXPLAIN SELECT max(colA), colA, MIN(colB), max(colB) FROM __mock_table_1 GROUP BY colA;">EXPLAIN SELECT max(colA), colA, MIN(colB), max(colB) FROM __mock_table_1 GROUP BY colA;</trans> <trans data-src="Agg { types=[max, min, max], aggregates=[#0.0, #0.1, #0.1], group_by=[#0.0] } | (__mock_table_1.colA:INTEGER, agg#0:INTEGER, agg#1:INTEGER, agg#2:INTEGER)">Agg { types=[max, min, max], aggregates=[#0.0, #0.1, #0.1], group_by=[#0.0] } | (__mock_table_1.colA:INTEGER, agg#0:INTEGER, agg#1:INTEGER, agg#2:INTEGER)</trans> <trans data-src="```">```</trans> <trans data-src="---">---</trans> <trans data-src="注意当表为空的时候，child的next不会返回tuples，aht_.">Note that when the table is empty, the next child will not return tuples, aht_</trans> <trans data-src="InsertCombine()就不会执行。">InsertCombine() will not be executed.</trans> <trans data-src="1. 如果查询中没有groupby，此时agg_executor->Next()仍要返回一些默认值，如函数`GenerateInitialAggregateValue()`中所指定的。">1. If there is no groupby in the query, agg_executor ->Next() still needs to return some default values, such as those specified in the function 'GenerateInitialAggregateValue()'.</trans> <trans data-src="为了复用`GenerateInitialAggregateValue(), InsertCombine()`这两个函数，我在处理空表时，对于`plan_->aggregates_`中使用到的各聚集函数，均构造一个null值作为agg_val，调用InsertCombine试图插入到hashtable中，此时`InsertCombine()`会对hashtable设置初值，而`CombineAggregateValues()">In order to reuse the two functions' GenerateInitialAggregateValue() and InsertCombine() ', when I process empty tables, I construct a null value as the agg_val for each aggregation function used in' plan_ ->aggregates_ ', and call InsertCombine to try to insert it into the hashtable. At this time,' InsertCombine() 'will set the initial value for the hashtable, while' CombineAggregateValues()</trans> <trans data-src="`会忽略掉我构造的这些null值，不动初值。">`These null values constructed by me will be ignored and the initial value will not be changed.</trans> <trans data-src="但这些只是aggregate函数们的默认值，如果查询语句有group by，那么还要构造这些列的空值作为默认值。">However, these are only the default values of aggregate functions. If the query statement has group by, then null values of these columns should be constructed as the default values.</trans> <trans data-src="2. 查询中有groupby，不返回值">2. There is groupby in the query, and no value is returned</trans> <trans data-src="---">---</trans> <trans data-src="在测试`select v4, min(v1) + sum(v2) + max(v3), count(*) from t1 group by v4;`">In Test ` select v4, min (v1)+sum (v2)+max (v3), count (*) from t1 group by v4`</trans> <trans data-src="的时候，aggregate_executor->Next()中的executed_无法等于true，导致这个Next()被无限调用。">The executed_ in aggregate_executor ->Next() cannot be equal to true, which causes the Next() to be called infinitely.</trans> <trans data-src="原因是我在Next()中修改hashtable后做了多余的更新迭代器的操作`aht_iterator_ = aht_.">The reason is that after I modified the hashtable in Next(), I did redundant operations to update the iterator ` aht_iterator_=aht_</trans> <trans data-src="Begin();`">Begin();`</trans> <trans data-src="导致无法满足结束的条件。">The end condition cannot be met.</trans> <trans data-src="只需在首次更新hashtable之后更新begin迭代器即可，即`Init()`和`if(!aggregated_)`中。">Just update the begin iterator after updating the hashtable for the first time, that is, in 'Init()' and 'if (! Aggregated_)'.</trans> <trans data-src="（在首次更新之前，hashtable为空，Begin返回的是end，这不是想要的）">(Before the first update, hashtable is empty, and Begin returns end, which is not what you want.)</trans>

[CMU15445] bustub 2023 Project2

• Author: rin
• Time: April 23, 2023
• Classification: Unclassified
• comment

<trans data-src="Codes: [b_plus_tree.cpp](">Codes: [b_plus_tree.cpp](</trans> <trans data-src="https://git.lo-li.net/rin/bustub/src/branch/master/src/storage/index/b_plus_tree.cpp">https://git.lo-li.net/rin/bustub/src/branch/master/src/storage/index/b_plus_tree.cpp</trans> <trans data-src=" "b_plus_tree.cpp")"> "b_plus_tree.cpp")</trans> <trans data-src="# Task1 B+Tree Pages "># Task1 B+Tree Pages </trans> <trans data-src="## Page types">## Page types</trans> <trans data-src="There are 4 types of headers (classes) named as `b_plus_*_page_.h`">There are 4 types of headers (classes) named as `b_plus_*_page_.h`</trans> <trans data-src="1. `b_plus_tree_page.h`">1. `b_plus_tree_page.h`</trans> <trans data-src="2. `b_plus_tree_header_page.h`">2. `b_plus_tree_header_page.h`</trans> <trans data-src="3. `b_plus_tree_internal_page.h`">3. `b_plus_tree_internal_page.h`</trans> <trans data-src="4. `b_plus_tree_leaf_page.h`">4. `b_plus_tree_leaf_page.h`</trans> <trans data-src="Only `internal_page` and `leaf_page` are inherited from the base class defined in `b_plus_tree_page`.">Only `internal_page` and `leaf_page` are inherited from the base class defined in `b_plus_tree_page`.</trans> <trans data-src="At first, I was confused by the words `RootPage` and `HeaderPage`. ">At first, I was confused by the words `RootPage` and `HeaderPage`. </trans> <trans data-src="In fact, `header_page` does not belong to the inheritance tree. ">In fact, `header_page` does not belong to the inheritance tree. </trans> <trans data-src="It stores the metadata of a `BPlusTree` object, which is `root_page_id` for now.">It stores the metadata of a `BPlusTree` object, which is `root_page_id` for now.</trans> <trans data-src="`header_page` always exists in an initialized `BPlusTree`. ">`header_page` always exists in an initialized `BPlusTree`. </trans> <trans data-src="But `root_page_id` may be `INVALID_PAGE_ID` at some time, meaning the tree is **empty** (tree exists but no data).">But `root_page_id` may be `INVALID_PAGE_ID` at some time, meaning the tree is **empty** (tree exists but no data).</trans> <trans data-src="---">---</trans> <trans data-src="## Page structure">## Page structure</trans> <trans data-src="Both 2 types of pages share the common member variables `page_type_`, `size_` and `max_size_` defined in `BPlusTreePage`.">Both 2 types of pages share the common member variables `page_type_`, `size_` and `max_size_` defined in `BPlusTreePage`.</trans> <trans data-src="But the data area `MappingType array_[0]` is defined in `leaf_page.h` and `internal_page.h` separately.">But the data area `MappingType array_[0]` is defined in `leaf_page.h` and `internal_page.h` separately.</trans> <trans data-src="This is because the array is a *Flexible array* and it must be placed at the end of an object. ">This is because the array is a *Flexible array* and it must be placed at the end of an object. </trans> <trans data-src="But in `leaf_page`, there is an extra member `next_page_id_` after common members. ">But in `leaf_page`, there is an extra member `next_page_id_` after common members. </trans> <trans data-src="The class structure is as below:">The class structure is as below:</trans> <trans data-src="```">```</trans> <trans data-src="|| page_type_ | size_ | max_size_ | next_page_id_ | array_....... ||">|| page_type_ | size_ | max_size_ | next_page_id_ | array_....... ||</trans> <trans data-src="|| page_type_ | size_ | max_size_ | array_....... ||">|| page_type_ | size_ | max_size_ | array_....... ||</trans> <trans data-src="```">```</trans> <trans data-src="In later codes, a pointer of `internal_page` will be `reinterpret_cast` to a point of `leaf_page`. ">In later codes, a pointer of `internal_page` will be `reinterpret_cast` to a point of `leaf_page`. </trans> <trans data-src="And `page_type_` will be accessed to check the type of this page.">And `page_type_` will be accessed to check the type of this page.</trans> <trans data-src="Due to the common memory structure in both pages' headers, this operation is safe.">Due to the common memory structure in both pages' headers, this operation is safe.</trans> <trans data-src="### Internal page">### Internal page</trans> <trans data-src="```">```</trans> <trans data-src="| KEY(0)+PAGE_ID(0) | KEY(1)+PAGE_ID(1) | KEY(2)+.... |">| KEY(0)+PAGE_ID(0) | KEY(1)+PAGE_ID(1) | KEY(2)+.... |</trans> <trans data-src="⬆">⬆</trans> <trans data-src="[INVALID]">[INVALID]</trans> <trans data-src="```">```</trans> <trans data-src="There are $$m+1$$ pairs of `<key, page_id_t>` in an internal page. ">There are $$m+1$$ pairs of `<key, page_id_t>` in an internal page. </trans> <trans data-src="The 0th pair has an invalid key. ">The 0th pair has an invalid key. </trans> <trans data-src="So an Internal Page stores **m** ordered keys and **m+1** child pointers. `">So an Internal Page stores **m** ordered keys and **m+1** child pointers. `</trans> <trans data-src="max_size_ == m+1`">max_size_ == m+1`</trans> <trans data-src="The assignment specification recommends:">The assignment specification recommends:</trans> <trans data-src="> split an internal node when number of values reaches `max_size` before insertion">> split an internal node when number of values reaches `max_size` before insertion</trans> <trans data-src="### Leaf page">### Leaf page</trans> <trans data-src="Every key in a leaf page is valid.">Every key in a leaf page is valid.</trans> <trans data-src="> split a leaf node when the number of values reaches `max_size` after insertion">> split a leaf node when the number of values reaches `max_size` after insertion</trans> <trans data-src="So in any stable situation, max_size_ < size_">So in any stable situation, max_size_ < size_</trans>

Manacher instance

• Author: rin
• Time: July 18, 2020
• Classification: Unclassified
• comment

<trans data-src="[前坑](">[Front pit]（</trans> <trans data-src="https://lo-li.net/1311-manacher-%E7%AE%97%E6%B3%95%E6%B1%82%E6%9C%80%E9%95%BF%E5%9B%9E%E6%96%87%E5%AD%90%E4%B8%B2.html">https://lo-li.net/1311-manacher-%E7%AE%97%E6%B3%95%E6%B1%82%E6%9C%80%E9%95%BF%E5%9B%9E%E6%96%87%E5%AD%90%E4%B8%B2.html</trans> <trans data-src=")">)</trans> <trans data-src="!">!</trans> <trans data-src="[manacher2.png](">[manacher2.png](</trans> <trans data-src="https://static.lo-li.net/typecho/2020/07/2154660342.png">https://static.lo-li.net/typecho/2020/07/2154660342.png</trans> <trans data-src=")">)</trans> <trans data-src="符号说明：">Symbol description:</trans> <trans data-src="* `i`为字符串中的下标，`S`是预处理后的字符串">*'i' is the subscript in the string, and 'S' is the preprocessed string</trans> <trans data-src="* `P[i]`代表以`i`为中心（含）的地方，回文的半径长度。">*'P [i]' represents the radius length of palindrome where 'i' is the center (included).</trans> <trans data-src="例如`P[1]=1`意为以`'#'`为中心的最长回文子串为`"#"`，半径长度为1；">For example, 'P [1]=1' means that the longest palindrome substring centered on '#' is' # ', and the radius length is 1;</trans> <trans data-src="同样，`P[16]=4`意为以`'x'`为中心的最长回文子串`"#p#x#p#"`的半径`"#p#x"/"x#p#"`的长度4">Similarly, 'P [16]=4' means the length 4 of the radius' # p # x '/' x # p # 'of the longest palindrome substring' # p # x # p # 'centered on' x '</trans> <trans data-src="* `id`代表当前已知的最长回文子串的中点；`">*'id' represents the midpoint of the longest palindrome substring currently known`</trans> <trans data-src="mx`代表该子串的半径长度+1">Mx ` represents the radius length of the substring+1</trans> <trans data-src="* 带圈数字代表步骤">*Circled numbers represent steps</trans> <trans data-src="①、起始状态，`i=0,mx=0,id=0`">① , Start Status, ` i=0, mx=0, id=0`</trans> <trans data-src="此时`i=1 >= mx=0`，即此位置已经超过了已知的最长回文范围（因为才开始嘛），同时也能确定`P[1]=至少=1`。">At this time, 'i=1>=mx=0', that is, this position has exceeded the known longest palindrome range (because it just started). At the same time, it can be determined that 'P [1]=at least=1'.</trans> <trans data-src="因此现在需要以`i=1`为中心，从其半径范围外两边开始逐一比对。">So now we need to take 'i=1' as the center, and start the comparison one by one from both sides outside its radius.</trans> <trans data-src="（两边就是`i±P[i]`）">(The two sides are 'i ± P [i]')</trans> <trans data-src="发现`'$'≠'a'`，扩展不下去了便停止，更新`id=1,mx=id+P[id]=1+1=2`">If '$' ≠'a '' is found, the extension will stop and update 'id=1, mx=id+P [id]=1+1=2`</trans> <trans data-src="②、`i=2,mx=2,id=1`">②、`i=2,mx=2,id=1`</trans> <trans data-src="此时`i=2 >= mx=2`，同样超过了已知的最长回文范围（注意`mx`是边界+1），同理能确定`P[2]=至少=1`。">At this time, 'i=2>=mx=2', which also exceeds the known longest palindrome range (note that 'mx' is the boundary+1). Similarly, it can be determined that 'P [2]=at least=1'.</trans> <trans data-src="继续扩展发现`'#'='#'`，则`P[2]++`，再继续无法扩展`$≠b`。">If you continue to expand and find that '#'='#' ', then' P [2]++'will fail to expand' $≠ b '.</trans> <trans data-src="更新`id=2,mx=id+P[id]=2+2=4`">Update ` id=2, mx=id+P [id]=2+2=4`</trans> <trans data-src="③、`i=3,mx=4,id=2`">③、`i=3,mx=4,id=2`</trans> <trans data-src="此时`i < mx`了，说明`i`在**已知最长的回文子串（id=2的那个）**的势力范围内了。">At this time, 'i<mx' indicates that 'i' is within the sphere of influence of * * the longest palindrome string (the one with id=2) * *.</trans> <trans data-src="那么根据对称性，`i=3`在前面就有一个“分身”`j=2*id-i=1`">According to the symmetry, 'i=3' has a "avatar" in front of it ` j=2 * id-i=1`</trans> <trans data-src="现在想求的`P[i]`就和`P[j]`有一种关系：`P[i]`=**可能**=`P[j]`，那么此时以`i`为中心的回文子串最右的下标就**有可能**扩展到`i+P[i]-1`，例如此时：">The 'P [i]' you want to seek now has a relationship with 'P [j]': 'P [i]'=* * may * *='P [j]', then the right subscript of palindrome substring centered on 'i' will * * possibly * * extend to 'i+P [i] - 1', for example:</trans> <trans data-src="!">!</trans> <trans data-src="[manacher1.png](">[manacher1.png](</trans> <trans data-src="https://static.lo-li.net/typecho/2020/07/3248822752.png">https://static.lo-li.net/typecho/2020/07/3248822752.png</trans> <trans data-src=")">)</trans> <trans data-src="`i+P[i]-1=3+1-1=3 < mx=4`，`3`仍处在**能保证对称的<span style="background-color:#ffe699">势力范围内</span>**，那么上面的“可能”变成“一定”。">`I+P [i] - 1=3+1-1=3<mx=4 `, ` 3 ` is still within the scope of influence of * * to ensure symmetry<span style="background color: # ffe699"></span>* *, then the "may" above becomes "certain".</trans> <trans data-src="（当然现在串还太短效果不是很明显，可以参考后面的⑥；同时“不可能”的情况请参考文末）">(Of course, the current string is too short, and the effect is not obvious. Please refer to ⑥ below; in the case of "impossible", please refer to the end of the article.)</trans> <trans data-src="④、⑤同理">④ ⑤ Same reason</trans> <trans data-src="⑥、`i=6,mx=10,id=5`">⑥、`i=6,mx=10,id=5`</trans> <trans data-src="`P[6]=P[4]=2`，现在能从下标`6-2=4`和`6+2=8`的位置开始对比，而对称区域内的都不用再比了。">`P [6]=P [4]=2 `, now you can compare from the subscript '6-2=4' and '6+2=8', and no comparison is needed in the symmetric area.</trans> <trans data-src="所以就利用了已知的信息跳过了很多次对比。">So we use the known information to skip many comparisons.</trans> <trans data-src="然后`'b'≠'a'`停止，`6+2=8 < mx=10`，说明仍然处于最长的那个子串内部，所以不更新`mx,id`">Then ''b' ≠'a '' stops, '6+2=8<mx=10', indicating that it is still inside the longest substring, so 'mx, id' is not updated`</trans> <trans data-src="……">……</trans> <trans data-src="如此便能算出所有的`P[i]`并且能得到最长的长度。">In this way, all 'P [i]' can be calculated and the longest length can be obtained.</trans> <trans data-src="------------">------------</trans> <trans data-src="“不可能”的情况">"Impossible" situation</trans> <trans data-src="换一个例子">Another example</trans> <trans data-src="!">!</trans> <trans data-src="[manacher3.png](">[manacher3.png](</trans> <trans data-src="https://static.lo-li.net/typecho/2020/07/31183235.png">https://static.lo-li.net/typecho/2020/07/31183235.png</trans> <trans data-src=")">)</trans> <trans data-src="此时`i=11,id=8,mx=14`">At this time, ` i=11, id=8, mx=14`</trans> <trans data-src="根据预判，`P[11]=P[5]=5`。">According to the prediction, 'P [11]=P [5]=5'.</trans> <trans data-src="但是下标`11+5=16 >= mx=14`，预判超过了**能保证对称**的部分，于是剩余的部分`14,15`就要和`8,7`暴力对比。">But the subscript '11+5=16>=mx=14', the prediction exceeds the part that * * can guarantee symmetry * *, so the remaining part '14,15' will be compared with '8,7' violently.</trans>

Codeforces Round #384 (Div. 2)

• Author: rin
• Time: December 15, 2016
• Classification: Unclassified
• comment

<trans data-src="~~这个灰名选手一直坚持他蓝名梦想~~">~~This grey player has always adhered to his dream of becoming a blue star~~</trans> <trans data-src="------------">------------</trans> <trans data-src="[A. Vladik and flights](">[A. Vladik and flights](</trans> <trans data-src="http://codeforces.com/problemset/problem/743/A">http://codeforces.com/problemset/problem/743/A</trans> <trans data-src=")">)</trans> <trans data-src="有$$n$$个机场，要从$$a$$飞到$$b$$">There are $$n $$airports to fly from $$a $$to $$b$$</trans> <trans data-src="每个机场有航空公司$$0$$或$$1$$，同航空公司机场间移动无费用，不同的之间为$$|ni-nj|$$，问从$$a$$到$$b$$最少花费是多少。">Each airport has an airline of $$0 $$or $$1 $$. There is no cost for moving between airports of the same airline. The difference is $$| ni nj | $$. Ask what the minimum cost is from $$a $$to $$b $$.</trans> <trans data-src="因为同公司无费用，所以如果$$a$$和$$b$$地相同的话就输出$$0$$；">Because there is no cost in the same company, $$0 $$will be output if $$a $$and $$b $$are the same;</trans> <trans data-src="如果不相同，那么从$$a$$一定能移动到一个地方，是$$0$$和$$1$$的交界，然后在交界换一家公司飞到b，花费是$$1$$">If they are not the same, then they must be able to move from $$a $$to a place where $$0 $$and $$1 $$meet, and then fly from another company at the junction to b at a cost of $$1$$</trans> <trans data-src="```c++">```c++</trans> <trans data-src="int main(){">int main(){</trans> <trans data-src="int n, a, b;">int n, a, b;</trans> <trans data-src="char aType, bType;">char aType, bType;</trans> <trans data-src="scanf("%d %d %d", &n, &a, &b);">scanf("%d %d %d", &n, &a, &b);</trans> <trans data-src="getchar();">getchar();</trans> <trans data-src="for (int i = 1; i <= n; i++) {">for (int i = 1; i <= n; i++) {</trans> <trans data-src="if (i == a) scanf("%c", &aType);">if (i == a) scanf("%c", &aType);</trans> <trans data-src="else if (i == b) scanf("%c", &bType);">else if (i == b) scanf("%c", &bType);</trans> <trans data-src="else scanf("%*c");">else scanf("%*c");</trans> <trans data-src="}">}</trans> <trans data-src="puts((aType == bType || a == b) ? ">puts((aType == bType || a == b) ? </trans> <trans data-src=""0" : "1");">"0" : "1");</trans> <trans data-src="return 0;">return 0;</trans> <trans data-src="}">}</trans> <trans data-src="```">```</trans> <trans data-src="------------">------------</trans> <trans data-src="[B. Chloe and the sequence](">[B. Chloe and the sequence](</trans> <trans data-src="http://codeforces.com/problemset/problem/743/B">http://codeforces.com/problemset/problem/743/B</trans> <trans data-src=")">)</trans> <trans data-src="初始有一个序列$$[1]$$">There is initially a sequence $$[1]$$</trans> <trans data-src="然后将它本身加到后面变成$$[1,1]$$">Then add it to the end to become $$[1,1]$$</trans> <trans data-src="再将还未使用的最小的正整数加到正中的位置$$[1,2,1]$$">Add the smallest unused positive integer to the middle position $$[1,2,1]$$</trans> <trans data-src="重复$$n-1$$次">Repeat $$n-1 $$times</trans> <trans data-src="问这之后第$$k$$个数字是啥">Ask what is the $$k $$number after this</trans> <trans data-src="由~~观察瞎猜脑洞~~，~~可以看出~~，这个序列代表从$$1$$到某个数，(二进制)从右数起第一个$$1$$的位置，比如">From~~to observe the brain hole~~,~~, we can see that~~, this sequence represents the position of the first $$$from the right to a certain number, such as</trans> <trans data-src="| 数（二进制） | 1的位置 |">|Number (binary) | 1 position|</trans> <trans data-src="| ------------ | ------------ |">| ------------ | ------------ |</trans> <trans data-src="| 1(1) | 1 |">| 1(1) | 1 |</trans> <trans data-src="| 2(10) | 2 |">| 2(10) | 2 |</trans> <trans data-src="|3(11)|1|">|3(11)|1|</trans> <trans data-src="|4(100)|3|">|4(100)|3|</trans> <trans data-src="|5(101)|1|">|5(101)|1|</trans> <trans data-src="|6(110)|2|">|6(110)|2|</trans> <trans data-src="|...|...|">|...|...|</trans> <trans data-src="那么第$$k$$位是啥，就是$$k$$的右起第一个$$1$$位置。">Then what is the $$k $$bit? It is the first $$$$from the right of $$k $$.</trans> <trans data-src="可以直接用glibc里的`__builtin_ffs()`函数">You can directly use the '__builtin_ffs()' function in glibc</trans> <trans data-src="考虑到$$k$$超`int`用其`long long`版本 ` __builtin_ffsll()`">Considering that $$k $$super 'int' uses its' long long 'version' __builtin_ffsll()`</trans> <trans data-src="```c++">```c++</trans> <trans data-src="int main(){">int main(){</trans> <trans data-src="long long k;">long long k;</trans> <trans data-src="scanf("%*lld %lld", &k);">scanf("%*lld %lld", &k);</trans> <trans data-src="printf("%d\n", __builtin_ffsll(k));">printf("%d\n", __builtin_ffsll(k));</trans> <trans data-src="return 0;">return 0;</trans> <trans data-src="}">}</trans> <trans data-src="```">```</trans>

1. one
2. two
3. Next »